Re: Help us make programming by voice possible in Linux

[Bill Cox <waywardgeek gmail com>]

Hi, Rao.

On Thu, Jul 8, 2010 at 3:29 AM, Nischal Rao <rao nischal gmail com> wrote:
> Hi Bill,
>
> What exactly do you mean by custom command? Can you please give an example?
> It might be easy to add this feature in VEDICS.

>From 1996 to 1999, I programmed entirely by voice.  I wrote 1,600
custom Naturally Speaking macros, most of which executed custom emacs
lisp code.  I was able to be very productive writing C code, do
various tasks in a bash shell in emacs, and all the other various
tasks I did daily on my Windows box.  In short, custom voice macros is
the single most important feature for programmers who can't type.  I
suspect it's the most important feature for anyone who can't type.

Basically, a custom macro is just a command consisting of a word or
phrase.  When a command is recognised, a set of keystrokes is
emulated, which can include mouse clicks.  So, for example, I was able
to point the mouse at a directory in a bash shell in emacs and say "Go
there".  Mouse click was the most common command, which I bound to the
"T" sound, so it would not strain my voice like "click" does.

The reason the few voice programmers who where active back then were
unable to share many of our macros with each other was that our jobs
were all different.  My macros were heavily C, bash, emacs, and
Windows oriented.  Another person I knew of had a Windows with
Naturally speaking, and some link to a Linux box where she actually
worked.  I don't think she used emacs, so our macros were mostly
incompatible.  We did share some macros, but there was never any
central repository for them.

Simon has a potential solution for this problem.  Users can create
their own "scenarios", which activate a set of custom voice macros.
This also means that there are fewer commands in the speech model at
any time, potentially improving accuracy.  Users can upload and share
scenarios.  So, for example, I could create an emacs-bash scenario, or
an emacs-C-mode scenario, and upload it to the Simon web site.
Another user could create a gnome-terminal scenario, or an
eclipse-java scenario.  For my machine, I just pick and choose the
scenarios I want, and I can customise them.  Scenarios should
automatically switch based on the active window, and applications need
a way to set active scenarios.

Ideally a programming by voice environment would execute commands
continuously, and the active commands would be selected based on the
cursor position.  So, for example, where identifiers are legal, words
would be concatenated with underscores or capitals, while in comments
or quotes, the general English model would be used.  However, this is
not needed to match the productivity of programming with Naturally
Speaking.  Last I checked, NS still requires commands to be
recognised, and then executed, and wont let you say them continuously.
 If I had that, I'd program that way, rather than typing, because it
would be faster.

Here's an example of how I use to code by voice.  The - characters are
where I was forced to pause and wait for the command to be recognised
and executed.  I would say "for loop - i equals 1 - semi - i less or
equal 10 - semi - i plus plus - open - printf - i equals percent d
slash n - add param - i"  Without the pauses, I can say that faster
than I can type it.  "For loop" creates a for loop with parens and
brackets and positions the cursor in the parens.  "open" creates a new
line and indents, printf creates printf(""); and positions the cursor
in the quotes, and "add param" adds a space and common after the
quotes and positions the cursor there.  The resulting code is:

for (i = 1; i <= 10; i++) {
    printf("i=%d\n", i);
}

For typical complex identifiers, I always had to say "cat" to
concatenate words with capitals.  So, I could say "db - cat - inst -
cat - get - cat - netlist - parens - inst", to generate
dbInstGetNetlist(inst).  Ideally, the system should have an identifier
mode so I could just say "ident db inst get netlist of inst" without
any pause.

Bill