Re: GTK+ Accessibility Proposed API

[Peter Korn <korn sun com>]

Hi Bill, Havoc,

Just a few points to add to this detailed examination of our proposed API
(thanks for all your work Havoc!):

> > ...
> > 
> >  The GaccessibleRole enum should have its members namespaced.
> >  So GACCESSIBLE_ROLE_ALERT, etc.
> >  ...
> >  Do we need this enum to be extensible? What about widgets added in
> >  other libraries or in enums? You might look at PangoAttributeType
> >  or the recent patch I posted for GtkIconSize to see how to do
> >  an "extensible enum" (GType is also such an enum).
> 
> You're absolutely right, I'll look at PangoAttributeType.

Both Role and State should be extensible.  We've found in the past that as the
GUI evolves, we get new widget types that are distinct and different, and so
should be presented to the user as such (especially the blind user, who cannot
see them).  Similarly with State - sometimes these new widgets can behave in
new and novel ways, and so be in new and novel states.

> ...
> >
> >  We would conventionally say get_n_children() rather than
> >  get_child_count(), I think. The _count() convention is sort of
> >  a Java-ism.
> 
> Caught me yet again ;-)

I fear I may be the real culprit for the Java-isms...

> > ...
> > 
> >  What is the meaning of the "children_changed" signal?

When you have transient children (e.g. in a spreadsheet-style sparse table),
you cannot register signal handles on the children.  However, you nontheless
way to be able to signal that a child has changed, so the assistive technology
can report that (e.g. a spreadsheet telling a user the new value in a cell, a
screen magnifier moving the magnified portion of the screen to include that
cell, etc.).

> > ...
> > 
> >  gaccessible_set_parent(): I think gaccessible_add() would be better,
> >  i.e. make the method on the parent, as with gtk_container_add().
> >  But again I'm unclear on why accessibles are in a tree structure,
> >  so I may be wrong on this.
> >  Can children be removed? What's the rationale for why or why not?
> 
> This is a seldom used feature, but when it is needed the model seems
> to be "set child's parent and send parent notification".  I'll confer
> with some of the other architects.

In our Java work we found that we needed our setAccessibleParent() method for
some unusual scenarios with flyweights and with wierd MVC situations (I think
specifically with the JComboBox control in Swing).  

It's not clear to me that those special situations apply in GTK+, though they
may apply elsewhere in other toolkits that may use this API (and of course they
may apply in GTK+ - the trouble is, we won't know 'till we implement this on
the widgets, which won't be finished 'till after the API freeze...).

> > gaccessibleeditabletext.h:

We should consider conflating gaccessibleeditabletext.h with
gaccessibleetext.h.  In the Java work we realized we missed editing support
that should have been in the base text interface, and so added
AccessibleEditableText.  There are a number of situations in which text isn't
editable (e.g. static text), in which case the editing/setting functions
wouldn't work (though there is not reason for Copy to not work, for instance).

> > ...
> > 
> >  However, I would suggest replacing this function with:
> >    get_selection_bounds (GaccessibleText *text, gint *start, gint *end);
> >  then people can get the selection by getting its bounds, then
> >  calling get_text(). Also, I'd replace
> >    get_selection_end()/get_selection_start() with the single
> >    get_selection_bounds() method.
> 
> We'd like to keep the first one atomic.  Your second point is a
> good one, we'll discuss.

Hmmm... I can see situations in which it might be quicker to get the bounds of
the selection and not the selection itself (e.g. just to answer the question
"do I have something selected").  I think Havoc's suggestion makes sense.

> > ...
> >
> >  Conceivably in C it's easier to have methods on GaccessibleHypertext
> >  that get that info, instead of fooling with a tiny
> >  GaccessibleHyperLink object that will get memory-leaked all the time.
> 
> I'll have to discuss this with the folks who designed the Java
> one, to see how the AT vendors are using it.

It's convenient to simply ask multiple questions of a link, or reference back
to them when displaying a list of links separate from the rest of the Hypertext
document.  But I don't see a strong reason why this has to be maintained in the
C edition.

> > ...
> > 
> >  Most of the other interfaces seem to lack "setters", such as
> >  set_icon_description(); why does this one have a setter while there's
> >  e.g. no set_hyperlinks(), set_actions(), and that kind of stuff?
> 
> Implementation.  In a lot of cases it's difficult to implement and
> may not be necessary, we are including the ones that experience has
> shown assistive technologies will need.

Traditionally with screen readers there have been no descriptions/text
associated with icons, and so they build their own hashed dictionary of them
and associated names with the entries.  In an API-supported world of
accessibility, you may still see external applications coming in after the fact
and labeling the icons, hence the set method.

> > ...
> > 
> >  I believe this class is to access what we'd call a "sheet", that is,
> >  a spreadsheet-like widget, while GtkTable is a container widget used
> >  to line up widgets in dialogs and such. So conceivably it should be
> >  renamed to GaccessibleSheet, though I'm not sure it's a big deal.  If
> >  it's also used for say GtkTreeView then I could very well be on
> >  crack.
> 
> Yep, it'll be used for GtkTreeView also ;-)

Hmmm... Maybe I'm on crack as well.  We don't use AccessibleTable on trees in
Swing.  If we rename it, another candidate is GaccessibleGrid.  

> > ...
> > 
> >  I don't understand the get_index_at, get_row_at_index methods. Why
> >  have flat index space in addition to the grid indices? Do these map
> >  to Gaccessible indices? (It's dawning on me that maybe the purpose of
> >  Gaccessible having children is to do these "compound" accessible
> >  objects, such as a table with cells - is that right? If so and we
> >  revise Gaccessible as I suggested to not use indices, these methods
> >  would become I guess get_position (table, Gaccessible *child, gint
> >  *row, gint *column)
> 
> Indices have proved to be important in the past to ATs.  I found the
> flat index space really weird at first also, especially in a
> spreadsheet, but I am assured that it's "really" what the AT folks
> want.

Often times an AT will simply want to walk a bit foward and back - get the next
thing, get the previous thing.  This is a standard paradigm, though it doesn't
apply well to tables/sheets/grids.  However, we wanted to make sure that if an
AT did that, it could easily related it to the appropriate row, column.

> > ...
> > 
> >  What does get_row_extent_at() do?
> 
> For cells that span multiple rows/columns you need these methods.
> HTML tables are the driving force behind a lot of the eccentricity
> here.

Since this is a general accessibility API for (potentially) all GNOME
applications, we want to make sure we can express table/sheet widgets beyond
just those presently in GTK+ today.  A number of table/sheet widgets provide
cells that can span, hence it's inclusion in the API.

> >  I don't understand the get_column_header() method, why does
> >  it return another table?
> 
> Again this one is an HTML table thing.  I agree that it looks weird,
> we will confer on whether it's absolutely necessary.

I think we don't need this level of flexibility...

> > ...
> > 
> >   This interface again has "setters", what are those used for? (I can
> >   imagine an accessibility app setting the text in a cell I guess, but
> >   why would a user change say a column title? Or, even if there are
> >   cases where they would, don't we need a mechanism to indicate
> >   whether or not they can? Since most column titles won't be
> >   modifiable by the user.)
> 
> In most cases the setters are for the application, not the user.
> Otherwise an app that uses some nifty icon for a column title will
> have trouble providing an alternate description of the column.

In some cases the cell and row headers may be flyweights, and so after you cons
them up you would want to set their descriptions.

> > gaccessiblevalue.h:
> >
> >  "Value" seems a bit vague, maybe "GaccessibleNumber"?

Perhaps "Range" makes more sense to you?  The idea here is that there are a
number of widgets that sport a current value within in a specified range (for
example, scroll bars), and the user will want to know both the value, and the
range.  Often times an assistive technology will take this and then calculate a
percentage and report that instead ("You are 34% through the document").

> > ...
> >
> > gtkaccessible.h:
> >
> >  Doesn't this need get_widget() and set_widget() methods? (they don't
> >  need to be in the vtable, just "final" methods that can't be
> >  overridden).
> 
> We don't want to expose set_widget, and for get_widget we are
> currently
> just using the widget member.  Should only be used by the
> implementation
> internals, probably.
> 
> We can add gaccessible_get_widget() easily enough, maybe it will be
> useful.

Yah... Ideally this should be opaque, as AT shouldn't need to ever go directly
to the widget.  It's a backdoor of course in case we miss something.  I
dunno... I'm really torn on this one...

> > GtkWidget changes:
> >
> >  Suggest storing the GtkAccessible in object data instead of
> >  in an instance member, so no space is used unless you're using
> >  the accessibility feature.
> 
> The accessibility feature will always be used, by every widget (except
> maybe some non-visible widgets, as previously described).
> However the heavyweight part of the implementation will not always be
> present.  Again, we really need this to be highly visible and
> exposed, which is not the case with object data.

Hmmm... The most important thing to be visible is the method, which must be
overridden in subclasses of GtkWidget.  Mind you, if we store it in the same
place by default, than it may make subclassing easier 'cause the rest of the
code (that returns it, etc.) will know where to look...

> > ...
> > 
> > gtkaccessiblecomponent.h:
> >
> >  I could use some docs on what this object represents and when/how
> >  it's used.
> 
> This is the subinterface that represents the "onscreen GUI component"
> aspect of an accessible.  It's needed in particular by screen
> magnifiers.

The issue here is that you may have non-visible interface components which you
want to represent in the Accessible interface tree.  So, rather than forcing
those components to contain information about their bounding rectangle, etc.
(which doesn't apply them), you put it into a subinterface which all widget
components implement.

Though we haven't come across any of these such interface components in
practice in Swing yet, we wanted the generality to support them (which wouldn't
have been possible once we put information about how the object is rendered
into the base Accessible object class).

> > ...
> >
> >  If set_cursor_type() changes the mouse pointer, it's not
> >  implementable without some larger GTK changes that I can describe if
> >  you want, I was trying to implement gtk_widget_set_cursor() the other
> >  day. If it does something else, probably you should explain it so
> >  we can figure out how to implement.
> 
> I'll talk to some folks who have worked with AT implementors.
> Apparently
> this is something they need, probably for screen magnifiers and low
> vision situations.  Maybe we can figure something out...

No, this is not needed here.  We can remove it just fine.

> >  set_enabled() would be set_sensitive() in GTK terms, and has the
> >  prototype for get_enabled(). Maybe you mean get_enabled()?
> >  If not then get_enabled() seems to be missing.
> 
> Prototype is wrong.  This is a messy one, you can do the
> get() via the GaccessibleState, but for implementation reasons there
> can't be a set_state().  I will confer as to why set_enabled() is on
> our requirements list.

We can get by without this one too.


Again, thank you Havoc for your very detailed read and comments.  This is
wonderful help!


Regards,

Peter