gdk_set_interest_spot() [was Re: Setting window icons]

[Peter Korn <korn sun com>]

Hi lupus,

> > ... in addition, at least in the Java
> > platform case, for objects that don't formally take the focus but are
> > nontheless where keyboard input is going, like the flyweight components in 
> > the Swing JTabbedPane and the table cells in JTable, additional events are 
> > exposed for those.  With this "standard" suite of events, screen 
> > magnifiers are able to track pretty much everything interesting in
> > applications.
> >
> > What are the scenarios in which you would want a specific interest spot 
> > beyond tracking caret and input location?
> 
> The focus on a GtkCList row, for example: there is no caret and no input
> there, but it's the area that the user is interested to if he changes the
> focus row with the keyboard. Also, caret movement is needed, but it depends
> also in what coordinate system it is reported: if it is in characters/lines
> in the text buffer it's useless for gmag; if it's is in pixel coordinates
> it's useless for gspeech. We need both, IMHO.

You know, the more I think about it, the more I'm concerned that a general
gdk_set_interest_spot() method may not be the right approach.  My question is
is this spot of interest to who and for what?  Different users and different
assistive technologies have different needs from the platform.  As you cite
above - magnification cares very much about precise global coordinate
information, while screen reading is far more interested in content, especially
relative to other content.  Some of the more sophisticated products for people
with learning disabilities (e.g. dyslexia) need both so they can highlight the
text on the screen as they speak it via a synthesizer.  Voice control and
dictation systems are very concerned with programatic ways of manipulating all
of the widgets, as are other alternate input systems (such as single-switch
systems, eye-gaze technology, etc.).  Many are also very concerned with the
different categories and states of the widgets - there may be no point in
putting the name of an inactive widget into a command-and-control speech
recognition grammer.  

So, coming back to your idea.  I think a better approach is to have a set of
events (or one PropertyChange event and a set of properties) that various
assistive technologies would listen for and react as is appropriate for them. 
With the Java Accessibility API today, screen readers listen for state property
change events globally, and thereby track changes in the active cell in the
Swing equivalent of a GtkCList.  Screen readers speak the cell contents when
the user moves to a new cell, and through the AccessibleTable interface can
speak the row and column info as well.  Screen magnifiers track the cell with
magnification by retrieving the bounding rectangle of the cell.  And while we
don't do it today, a recognition grammer could potentially be altered if we had
a way of exposing the kinds of data that are acceptable to enter into that cell
(e.g. is the cell a formula in a spreadsheet or a numeric value?).  

By having a relatively rich set of standard events supported across all widgets
uniformly, we can leave the specific behavior and "user interface" presentation
up to the assistive technology.

On the caret issue you raise above -> with the Java Accessibility API, the
idiom is to simply say that the caret property changed and provide the old and
new offset locations of the caret in the text field (along with, of course, a
reference to that text field).  Subsequent Accessibility API calls would then
be made to retrieve the letter/word/sentence/line of text containing the caret,
as well as the bounding rectangle of any character at any offset (e.g. the
bounding rectangle of the charater to the right of left of the caret so it can
be magnified properly).  With this approach, you don't need to decide for all
times what information must be in each event - you let the assistive technology
retrieve what it needs in response to the event.


Peter