GNOME.org
 

Re: HFOSS Visual Audio

Hi All:
The HFOSS internships are only 10 weeks and are meant for relatively inexperienced college students. Based upon my experiences with this kind of thing in the past, we need strong mentors and well defined achievable tasks for this to succeed. In the HFOSS case, I believe we're lucky in that the students will have their professors for some mentoring, but we also need strong mentors from the GNOME side.
Given this and the capabilities of the people involved, I could see this internship taking a couple different paths: 

1. SURVEY/RECOMMENDATION:
-------------------------
Create a survey/recommendation for what can be done in GNOME to bring state of the art support for visual audio. This would include looking at existing commercial solutions (e.g., OSX, Windows, GNOME), solutions currently in research (e.g., TRACE, WGBH), and discussions with end users. The browser itself, such as Firefox, might also be viewed as a space independent of the OS.
The survey may also involve breaking the space up into different areas, such as handling streaming media vs. discrete notifications. In addition, some analysis of the technical infrastructure in place for GNOME (e.g., gstreamer, the notification-daemon, etc.) would be useful.
The deliverable would be a detailed survey with recommendations for what GNOME, and perhaps the industry as a whole, can do. 10 weeks is not a lot of time for this, but might be something achievable for a motivated and outgoing student. From there, we cross our fingers and hope someone will be there to actually implement the recommendations -- maybe it's something that a motivated professor might use as the core of a grant proposal. 


2. HACK/PROTOTYPE
-----------------
This one basically skips the formal survey idea above and jumps right to prototyping specifically in the notification/discrete-event space (i.e., not streaming media). Look at what the GNOME desktop has now, such as gstreamer, notification-daemon, the "alerts and sound effects" feature, and how these all integrate with apps like Thunderbird, Pidgin, etc. Work with Bryen to come up with a written design for ways to present these kinds of things visually. Then hack away at some specific solutions, such as Pidgin, Thunderbird, the Battery/Power Monitor, the Network Monitor, etc.
With this solution, Bryen can act as a mentor from the end-user perspective. We'd need help from a someone familiar with the way GNOME does audio and the way notifications happen, but a motivated student/hacker might be a self-starter.
The deliverables would be a design/style for how to handle visual audio for the notification/discrete-event space, specific examples for some applications, and perhaps a framework for making it easy to support this kind of thing from any application. The framework might also include considerations for integration of things such as closed captioning from streaming media.
The benefits of this task are that something actually gets done and it might spur on new ideas. The risk is that it may end up completely missing the mark for the best way to handle this space. 

Will

Bryen wrote:

On Thu, 2009-02-19 at 13:58 -0800, Peter Korn wrote:

Bryen,
It's nice to see thoughts & proposals around hearing impairments! Regarding your proposal - either as part of an updated proposal, or as part of the first work you proposed to do, I would like to see: (1) a review of the state of the art in other desktops (Mac, Windows, KDE) for visual events, and (2) discussions/connections with places like the TRACE Center & the Nation Center on Accessible Media (NCAM) at WGBH TV around their research in this area. Not that we should simply do as others do, but we should be informed by what others are doing and thinking in this space before we implement something. 



Well, the inspiration for this proposal actually comes from OSX where
they have the built in screen dimming flicker functionality if you
enable it.  I don't have access to OSX at the moment, so I can't say
where exactly you can enable it.

On the Windows side, there were some addons I could download to help
notify me when email came in.  One particular app was imap-notify.  In
addition to the sys tray application, it also provided a popup window in
the center of my screen which was very much a benefit to me with my
visual impairment (I have no peripheral vision.)

It is the OSX component that I wanted to not only emulate, but expand
upon with my suggestions for plugins.

I don't have access to Windows or OSX these days.  However, you do
(judging from the email you sent, heh), so perhaps you could look around
and see what XP or Vista has these days and let us know?
On the technical side, in addition to the issues you cite around UNIX audio sub-systems, there is also the question of video functionality that would enable some of the user interface you suggest (e.g. screen dimming). Technologies like Compiz might be a lovely way to do something like this, but then we run into the fact that not every desktop has Compiz (and if you use some other technology, will it be compatible with a Compiz desktop). I think this facet should also be explored - either in preparation to the proposal submission or as work to be funded under the project -> at least to the level of a survey and recommendations. 

Absolutely.  We have to recognize that there are factors that exist.
Geographic locations where regions may not have more modern machines,
economic issues where unfortunately, people with disabilities tend to
have less buying power.

And then there's the technical problems.  Compiz is a great solution but
it doesn't always work, even if you have the supported hardware.  I want
something that is definitely agnostic and not dependent on having
additional components.  And I want ease of use.  So usability factors
come into consideration here as well. 
So, I guess an additional question here is, should we be discussing the
"ideal look and feel" of such an app before it gets taken up by HFOSS?
Interns only have 3 months to work on this and it might behoove everyone
if we did a preliminary discussion of best look and feel so the interns
can dive in and get their hands dirty under the hood. 
Or perhaps HFOSS would relish the opportunity to give the intern a "From
Conception to Implementation" project.  I don't know.

One other thing.  We will need technical mentors for this.  I will
gladly and excitedly be the usability mentor, but I'm not qualified to
be a technical mentor.   Volunteers please?


Regards,

Peter Korn
Accessibility Architect & Principal Engineer,
Sun Microsystems, Inc.


This is my proposal for visual audio events as a project for the
upcoming HFOSS Summer Program.  We'll call it "Visual Audio" for now but
probably we should come up with a formal title at some point.

I would appreciate comments before we submit it formally to HFOSS.

The Situation:  While open source has done much to reach out to people
with accessibility needs, one group that hasn't had much attention is
Deaf or hearing-impaired users.  In particular, sound events, such as
incoming instant messages, IRC chat beeps, etc. are not heard.

____________________________________________

The Proposal:
Other operating systems have come up with solutions such as creating a
screen dimming flickers whenever an incoming message appears.

GNOME/X has some rather rudimentary functions that currently exist.  There is a system alert which flashes the screen and there is of course, the notification daemon.  Neither option works very well as it is not easily configurable.

I'd like to see us implement a similar feature for the GNOME Desktop
Environment, but take it a few steps further.

1)  Allow users to determine what applications send visual events.  If I
was listening to music, I wouldn't want visual effects happening
constantly. 
2)  Allow for customization of visual event effects. This is important,
because like myself, 10% of the Deaf population also lives with Usher
Syndrome (visual impairment.  I think the best approach is to create a
plugin type environment where the general community can contribute by
creating unique effects.  Examples would be:
--Screen dimming flicker
--Hard screen flicker
--Running lights around the border of the monitor
--Graphic popup in designated area of screen  (for me, I miss having
events pop up in the middle of my screen like on Windows.)
--Animated events, such as a snowball splat.  Sounds crazy, but its a
fun approach.

This proposal benefits not only Deaf users, but also the general
population of users who do not have speakers nor wish to enable their
speakers (e.g. in the workplace.)  So it has broad appeal.  Furthermore,
the "eye candy" effect of this application would help to further propel
adoption and mass creation of visual effect plugins.  As such, the
plugin environment should be easy for people to work with.

There are some technical considerations we should also cover.  For
example, Alsa vs. Pulse Audio.  Metacity vs. Compiz.  This app should be
able to function regardless of specific environment factors.

Possibly we could consider tying this into the notification daemon, thus reducing the work that the HFOSS interns have to do with building something from scratch.  There is of course one drawback in that not all applications use the notification pop-up bubbles.  How to work around this is another question to be dealt with.

Feedback on this topic is greatly appreciated.




_______________________________________________
gnome-accessibility-list mailing list
gnome-accessibility-list gnome org
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-list
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]