Re: [Kde-accessibility] focus tracking

[Joseph Scheuhammer <clown utoronto ca>]

Hi all,

On 01/07/10 3:53 PM, Jos Poortvliet wrote:
> Ok, thanks all for the responses. A couple of questions if you don't mind, to check I understand this properly:
> - AT-SPI does not have a way of tracking focus for magnification purposes?
> - So Gnome-Mag has developed an API for this, but it's currently CORBA based and thus needs changing
> - apps need to support this API for magnification to properly work
>    
Here is my understanding.

AT-SPI has a way of tracking focus, yes, but not specifically for 
magnification.  There are focus events that can be listened for, and, I 
believe, one can use AT-SPI to query where focus is.  The result is 
sometimes in terms of a GTK+ widget, in the sense that AT-SPI tells you 
that a specific widget has keyboard focus.

Gnome-mag has not developed an API for focus tracking per se.  Gnome-mag 
*is* a CORBA service, and is being ported to a DBus service.  All 
services have an API so that other processes can make use of them.   One 
of the functions the magnifier service provides is along the lines of 
"given these screen coordinates, place the centre of the magnified view 
at that point".  By itself, this function doesn't know anything about 
focus. However, something that does know where focus is can determine 
the relevant coordinates and tell the magnifier where to place its 
view.  (Aside:  gs-mag also implements the same DBus service).

What is the something that knows where focus is?  As far as I know, the 
only application that computes this is Orca.

And now for some historical background:  when I started the 
implementation of a magnifier within GnomeShell, my intent was to use 
AT-SPI to track keyboard focus, and have the magnifier update its view 
accordingly.  I proposed this to Will Walker, the lead Orca developer at 
the time.  His response was (my emphasis):

> So, we could deep dive on this whole thing and duplicate what Orca is 
> doing in Python to track the notion of "focus", but in Javascript, 
> requiring you to learn a whole new technology, a whole new API, and 
> *re-solve all the same problems Orca has solved*.  Or, you could 
> expose a simple API, which will be needed anyway, and let Orca do the 
> driving. ;-)

In other words, Orca is tracking keyboard focus.  Furthermore, my 
impression is that Orca is doing something above and beyond what AT-SPI 
provides; that Orca uses a host of heuristics for focus tracking.  
What's happening is a sequence like this:

1. User moves keyboard focus.
2. Orca uses AT-SPI and Orca's own built-in heuristics to determine the 
region on screen that contains the focussed element.
3. Orca instructs the magnifier service to update its view given the 
coordinates of that region.

In summary, the magnifier is pretty dumb when it comes to keyboard 
focus.  The genius here is Orca.

In closing, here is something of a proposal:  the machinery that Orca 
uses to track keyboard focus is not inherent to a screen reader.  Other 
processes could well make use of Orca's focus tracking machinery.  It 
would be very useful if these heuristics could be carved out of Orca and 
published as a separate module, library, or service that any process 
could make use of.

Hope that helps.

--
;;;;joseph

'Clown control to Mao Tse Tung.'
 - D. Bowie (misheard lyric) -