Re: Speech enabling GnomePanel

[Bill Haneman <Bill Haneman Sun COM>]

Jaideep:

I would like to second what Peter said; I'd recommend looking at at-poke
and at-spi in general.

Rather than speech-enabling the panel (only), since AT-SPI provides a
command and control API for the GUI via AccessibleAction, why not plug
your Sphinx4 system into the AT-SPI interfaces?  That way, everything in
GNOME which properly implements AT-SPI interfaces will be controllable
via speech.

I think the rough outlines of what you may wish to do:

(1) track "focus:" and "window:" events via at-spi's
AccessibilityRegistry listener interface, to determine the user's
current application context;

(2) when the context changes, use AT-SPI to search the current hierarchy
for objects with SPI_ROLE_MENU, SPI_ROLE_MENUITEM, etc. or for things
that implement AccessibleAction (see cvs module GOK, gok/gok-spy.c for
examples of how to do that);

(3) use Accessible_getName to get the names/labels on the menu items or
other actionable things, and use them as the "dictionary" or
"vocabulary" of currently available commands to match against in
Sphinx4;

(4) when you recognize a match (i.e. the user has spoken the name of a
menu or menu item), use AccessibleAction_doAction() to invoke the
appropriate response from the GUI.

best regards and good luck!

- Bill

Jaideep wrote:

> Hi,
> I am currently doing a project which will add a speech
> enabled interface to the GNOME panel. It will be a
> command and control system allowing the user to
> control the panel menu/submenus through speech. The
> ASR engine that I have decided to use is Sphinx4. In
> order to provide its services to GNOME what are the
> steps to be takem? I have just read about the AT-SPI,
> I want to read more about it. Can anybody give some
> direction? Any suggstions will be of great help.
>                                               Thanks,
>                                           Jaideep.