Re: HFOSS Visual Audio

[Peter Korn <Peter Korn Sun COM>]

Bryen,

It's nice to see thoughts & proposals around hearing impairments! 

Regarding your proposal - either as part of an updated proposal, or as 
part of the first work you proposed to do, I would like to see: (1) a 
review of the state of the art in other desktops (Mac, Windows, KDE) for 
visual events, and (2) discussions/connections with places like the 
TRACE Center & the Nation Center on Accessible Media (NCAM) at WGBH TV 
around their research in this area.  Not that we should simply do as 
others do, but we should be informed by what others are doing and 
thinking in this space before we implement something.

On the technical side, in addition to the issues you cite around UNIX 
audio sub-systems, there is also the question of video functionality 
that would enable some of the user interface you suggest (e.g. screen 
dimming).  Technologies like Compiz might be a lovely way to do 
something like this, but then we run into the fact that not every 
desktop has Compiz (and if you use some other technology, will it be 
compatible with a Compiz desktop).  I think this facet should also be 
explored - either in preparation to the proposal submission or as work 
to be funded under the project -> at least to the level of a survey and 
recommendations.


Regards,

Peter Korn
Accessibility Architect & Principal Engineer,
Sun Microsystems, Inc.

> This is my proposal for visual audio events as a project for the
> upcoming HFOSS Summer Program.  We'll call it "Visual Audio" for now but
> probably we should come up with a formal title at some point.
>
> I would appreciate comments before we submit it formally to HFOSS.
>
> The Situation:  While open source has done much to reach out to people
> with accessibility needs, one group that hasn't had much attention is
> Deaf or hearing-impaired users.  In particular, sound events, such as
> incoming instant messages, IRC chat beeps, etc. are not heard.
>
> ____________________________________________
>
> The Proposal:
> Other operating systems have come up with solutions such as creating a
> screen dimming flickers whenever an incoming message appears.
>
> GNOME/X has some rather rudimentary functions that currently exist.  There is a system alert which flashes the screen and there is of course, the notification daemon.  Neither option works very well as it is not easily configurable.
>
> I'd like to see us implement a similar feature for the GNOME Desktop
> Environment, but take it a few steps further.
>
> 1)  Allow users to determine what applications send visual events.  If I
> was listening to music, I wouldn't want visual effects happening
> constantly.  
>
> 2)  Allow for customization of visual event effects. This is important,
> because like myself, 10% of the Deaf population also lives with Usher
> Syndrome (visual impairment.  I think the best approach is to create a
> plugin type environment where the general community can contribute by
> creating unique effects.  Examples would be:
> --Screen dimming flicker
> --Hard screen flicker
> --Running lights around the border of the monitor
> --Graphic popup in designated area of screen  (for me, I miss having
> events pop up in the middle of my screen like on Windows.)
> --Animated events, such as a snowball splat.  Sounds crazy, but its a
> fun approach.
>
> This proposal benefits not only Deaf users, but also the general
> population of users who do not have speakers nor wish to enable their
> speakers (e.g. in the workplace.)  So it has broad appeal.  Furthermore,
> the "eye candy" effect of this application would help to further propel
> adoption and mass creation of visual effect plugins.  As such, the
> plugin environment should be easy for people to work with.
>
> There are some technical considerations we should also cover.  For
> example, Alsa vs. Pulse Audio.  Metacity vs. Compiz.  This app should be
> able to function regardless of specific environment factors.
>
> Possibly we could consider tying this into the notification daemon, thus reducing the work that the HFOSS interns have to do with building something from scratch.  There is of course one drawback in that not all applications use the notification pop-up bubbles.  How to work around this is another question to be dealt with.
>
> Feedback on this topic is greatly appreciated.
>
>
>
>