Re: [Kde-accessibility] Questions regarding AT-SPI and ATK

[Peter Korn <peter korn sun com>]

Hi Gunnar,

> The KDE Accessibility Project is currently elaborating on what is the best 
> way to add AT-SPI support to Qt and KDE. Clearly this will be done by 
> using the Qt Accessibility Architecture (which currently gets extended by 
> Trolltech). However, there are a number of questions that IMO need to get
> solved:
> 
> 1. What are the exact events that are used in AT-SPI and ATK?
> 2. Is there a possibility to determine whether there is an assistive
> technology?
> 3. Is there a possibility for applications to do mouse (and keyboard) 
> grabs without stealing the AT applications mouse clicks?
> 4. As far as I have heard AT-SPI supports custom interfaces if an 
> application wants to provide information that is not handled within the
> AT-SPI standard.
> How exactly does this work?
> 5. What are the exact dependencies of the required party of the AT-SPI
> architecture?
> 
> I will give some more details (and some background) for these questions:
> 
> 1. In order to extend their accessibility framework, the Trolltech 
> developers need to know which events are used in AT-SPI and ATK and which 
> details are used to describe these events. As far as I looked there is no 
> documentation about the event types in AT-SPI. In ATK the documentation 
> for the events is spread over all interfaces. So far I found the
> following events:
> 
> Key press, Key release, Children change, Focus change, Property change,
> State change, Visible data change, Selection changed, Column deleted,
> Column inserted, Column reordered, Model changed, Row deleted,
> Row inserted, Row reordered, Text attributes changed, Text caret moved,
> Text changed, Text selection changed
> 
> Is this list complete? Which information describing those events is 
> included in these events?

I will leave it to BAUM and UToronto (and the Dasher folks) to detail which
of these events they use.  By far the most used are:

 focus change
 property change
 state change
 selection change
 text caret moved
 text changed
 text selection changed

There should also be a text inserted event.  Visible data change and text
attribute changed are also used a fair amount, but I think to a lesser
extent.

> 2. In some cases preparing the events can be expensive. Therefore the Qt
> Accessibility Architecture contains a method that returns true if there 
> are AT clients, and false if there are no AT clients. Clearly the easiest
> implementation for this method when bridging to AT-SPI is to always return
> true if the application got registered with the broker. On the other hand 
> it might be worth to do a more sophisticated implementation. For that we 
> need to be able to ask the broker whether there are AT clients. In that 
> case we also have to make sure that it is save to turn the sending of the 
> events off when there is no AT client.

In GNOME there is a per-user setting for whether accessibility API support
should be on or off, and both Gnopernicus and GOK will warn the user if the
flag is turned off when the they are launched.  This is stored in the user's
gconf database.  In StarOffice/OpenOffice.org (for example), if the setting
is on then a Java VM is invoked to be (part of) the bridge.  You might use
that idiom.

> 3. One of the KDE developers wrote a prove-of-concept application that 
> uses the current Qt Accessibility Architecture. This application is called
> Klaviatura and serves for two purposes: It can be used to remotely control
> the menu bars, tool bars and popup menus of KDE applications, and it 
> contains an on-screen keyboard. Technically Klaviatura consists of a 
> plug-in for Qt and an external application. Both parts communicate with
> each other via DCOP.
> 
> When writing this application the developer found that applications often 
> make mouse grabs when showing pop up menus. If then a mouse click was 
> intended to go to an assistive technology it will be used by the wrong
> application.
> 
> In Klaviatura this problem is solved as follows:
> The plug-in intercepts the event handling in order to be able to process 
> mouse clicks before the application processes them. If the mouse click was 
> directed to a window of Klaviatura that mouse click is intercepted and 
> sent to the Klaviatura application (via DCOP). Otherwise the mouse click 
> is not intercepted (and therefore will be processed by the application).
> 
> There are multiple possibilities for solving this issue in AT-SPI:
> - - Avoid popup menus (and other things that require mouse grabs). This is 
> not a good solution as it interferes with the current user interface 
> design of many applications. Not showing a popup menu means either less 
> functionality or (in the case that you need an assistive technology to 
> activate an entry in the popup menu) less usability.
> - - Do the popup menus without mouse grabs. This is not a good solution as 
> pop up menus need mouse grabs in order to work properly.
> - -Send all mouse clicks to the ATs first and ask for a permission to 
> process them.
> - -Send those mouse clicks that are not directed to an application window 
> to the ATs and ask for a permission to process them.
> 
> The last two possibilities sound acceptable to me. I am not sure which is 
> the better one. Many assistive technologies might not be interested in 
> processing all mouse clicks, but there might be some few ATs that need top 
> process them all.
> 
> Solving this problem will be important as some KDE developers will 
> otherwise have objections against AT-SPI.

There is yet another approach, which is being pursued by Sun in
communication with folks in X.org and the XFree86 community - a new X
extension that provides all keystokes and mouse events just after
XKB/AccessX and just before anything else.  An AT tool would register for
the keystrokes it cares about, and either listen and let them contine on, or
consume them.  The problem with this approach is that it will take time for
the X extension to be adopted and deployed.

> 4. Some KDE developers have concerns that AT-SPI is not powerfull enough. 
> In order to convince them that we will be able to extend the protocol if
> necessary we need support for custom interfaces.

AT-SPI is explicitly designed to be extensible.  That's why we have
sub-interfaces for things like text, tables, etc.  In the Java predecessor
to AT-SPI, we evolved the interface several times (tables & relationships
are two notable examples of things that weren't in the first release).

Are there specific questions about how to evolve AT-SPI?

> 5. In order to get an acceptable solution for KDE we have to evaluate the
> dependencies of those parts of the GNOME Accessibility Architecture that 
> will be used within KDE. When looking at ATK and the broker I fear that 
> both will most likely have too much dependencies. However, as we currently 
> do not have a clear position of KDE to AT-SPI, so I cannot say which 
> dependencies will be OK and which have to be changed.
> 
> If what I heard is correct the ATK dependencies are due a dependency on 
> GAIL which was introduced in order to force the user to have a working
> architecture once ATK is installed. This dependency on GAIL pulls a
> dependency on GNOME into ATK and should be avoided if we decide to use ATK 
> in KDE.

Hmmm... Perhaps Padraig or someone else can speak to this, but I didn't
think that ATK depends upgon GAIL.  The reverse is true.

> The broker currently requires GTK+2. From a KDE perspective this 
> dependency is too much. When looking from outside on the broker one would 
> think that dependencies to glib and ORBIT2 would be enough.
> 
> Also the CORBA dependency might turn out to become a problem for many KDE
> developers.
> 
> In either case I hope that we will find a clear position of KDE on the KDE
> Developers Conference at the end of August.


Regards,

Peter Korn
Sun Accessibility team