Re: Justification for the button order change - the long version

[Seth Nickell <snickell stanford edu>]

> First: I do not believe that it is correct to say "with left-right,
> top-bottom readers, your eye is left resting on the lower right corner
> of a window when you are done reading text". Because of the way we read,
> when we finish one line (whether it is the last line of text or not) it
> is our natural tendency to bring our eyes back to the left side. What
> Gnome2 will be doing is actually going against this. A previous post in
> this thread linked to Sun's "Dialog Box Design" pages - on the second
> image on that page, Sun pretty much explicitly diagrams this process.
> 
> http://java.sun.com/products/jlf/ed2/book/images/dialog.stdTabTrav.gif

As I said, this particular reason for the GNOME2 button ordering is a
contentious point. All else being equal, I tend to trust the Apple HIG
more than the JLF guidelines, but given conflicting "official" sources
we are left arguing over our intuitions, where we clearly disagree and
it is doubtful we will convince the other. I probably shouldn't have
mentioned the point, but the damage is done.

> Second: Where in all this (and I include in this the archived
> discussions we were pointed to earlier) is the actual human testing?
> When Sun did its HIG work on Gnome, they actually plopped people
> (non-hacker types) in front of computers and got their feedback. If that
> has been done in this case, it certainly isn't obvious - and I'd have
> thought it would've been brought up by now since that would be the
> killer argument in support of making this sort of change.

I'm really getting sick of this argument. I user test a lot of things,
but it is hard work, not particularly fun, time consuming, and not
feasible for every (or even a significant minority) of the suggestions
in the HIG. If you would like to pay me to perform this testing, I'll be
much more open to it. Its not a trivial process *at all* (just think
about the difficulty in getting a significant number of people to come
in and be tested! and that's aside from having to design good tests,
analyzing results, etc etc. I think people I know are avoiding me when I
have that user testing look in my eyes now ;-)

Additionally, long term productivity is one of the hardest things in
interface design to test, precisely because of the "long term" clause
;-) This makes the amount of difficulty in mounting a study tremendously
compounded.

I did some testing that relates to this a few weeks ago, but I wasn't
really concentrating on button ordering, though it was one of a number
of things I was looking for, and it was not a rigorously controlled
experiment but a "real world observation" (I'm cutting the observations
that weren't related to button order or this thing would be 10 times as
long ;-):

Given that MacOS already uses this button ordering, we have a perfect
test case. In the Stanford environment it is installed on cluster
machines, and various academic/administrative buildings and clusters,
but most people own Windows PCs, thus it is a minority use situation in
a heterogenous environment.

Observed 8 people for 30-45 minutes each (you do the math: this stuff
takes a freaking long time, now look at the size of the HIG and try to
figure out how long it would take to test even 5% of the suggestions
therein, and this really wasn't a very adequate test). Breakdown was as
follows:

2 were freshman working on scanning and editing drawings/photographs.
3 were working on "compound" documents of some sort (I ignored people
that were just typing, since there wasn't much to see), one was a junior
and two were freshman.
1 was a sophomore working in Mathematica
1 was a freshman browsing the web and reading her e-mail
1 was a junior who reported having virtually no Macintosh experience at
the time of the test who I asked to perform a series of tasks intended
to test launching and working between multiple applications

All but two people reported having very little interaction with MacOS
prior to coming to Stanford, and 7 out of 8 people had Windows PCs in
their rooms.

The long and short is that the first six people appeared to have no
difficulty working with the dialogue boxes, either in making mistakes or
in fluency (I didn't observe any misclicked dialogues). Four of the six
appeared to hardly read standard dialogues (three of whom reported using
the cluster anywhere from 1 to 5 hours a week, the other reported only
"occasional" use) and went immediately to the button they wanted. In
less standard dialogues their responses seemed fairly normal in speed.

Of the remaining two, the junior who a recruited with no experience read
dialogues carefully before making selections, which you would expect on
a new system anyway before you are familiar with the text. He made no
errors in dialogue buttons (though he did make a number of other
mistakes with the system), though he was generally plodding through the
interface as a whole, so very little quick response decision was being
made.

The freshman reading her e-mail had an kerberos ticket authentication
dialogue pop up that suprised her. She typed in the authentication
information quickly, but seemed to hover over the buttons for a few
seconds. After she made her selection I asked her why she had paused.
She reported that she was "not sure I was supposed to log in again, so I
wondered if I should cancel", which makes it appear not to be a dialogue
button error.

In post session debriefing I asked all participants to list complaints
they had with MacOS. Nobody mentioned dialogue button order, though one
person did note that they disliked that MacOS dialogues tend to be very
modal and can't be dragged around.

When specifically prompted what they thought of the dialogue button
ordering, and shown an example of a Windows file open dialogue's buttons
contrasted with Macintosh, 6 people said they had never noticed, had no
preference, and it didn't bother them (including the new junior user), 1
person said they preferred Windows but hadn't noticed and it didn't
bother them, 1 person said they had noticed and preferred Macintosh, but
Windows' ordering didn't bother them. Nobody said they were bothered by
the MacOS button order (or the Windows button order for that matter).
When prompted, no users could remember instances where they thought
having the switched dialogue orders had caused them problems.

This could be taken as an argument that it simply doesnt' matter.
However, I'm inclined to say that while the confusion effect would be
noticable to users, the gains from having a different order is not
something users would necessarily be sufficiently conscious of to
report. That does not mean it is insignificant. But that is my somewhat
slanted interpretation of the results. 

Either way, I think it bears out the argument that this isn't a tragic
change to GNOME. I also assumed that since for most of these users their
primary computer was Windows, they are also not having problems with
Windows as a result of using MacOS, but I did not verify this at the
time. 

Its worth mentioning that the contextual difference between MacOS and
Windows is probably larger than GNOME1 -> GNOME2. However, its also
worth pointing out that most users reported being quite familiar with
the Windows version of the particular application they were using (Word,
Photoshop, Internet Explorer, Netscape (there were also some MacOS
specific apps used)), and hence presumably encountered dialogues they
already had some memory of, but still had no trouble.

> Third: It is not a case of "doing what Windows does" or "not doing what
> Windows does"; or at least it shouldn't be. It should be a matter of
> things working the way people expect them to work, whether or not that
> just happens to be how Windows does it. The bottom line is that most
> people expect the default apply/yes/okay button to be on the left. BTW
> the "previous/next" button discussion has very little to do with this;
> if you think about how people organize sequential events you'll realize
> why the traditional ordering of this particular button pair makes
> perfect sense.

What is the traditional ordering? You mean having "[previous] [next]"
??? If that's what you're saying, I agree fully, and part of what I want
to do is preserve the consistency of that sort of dialogue.

> Interaction with dialog boxes should require as little thought as
> possible, because they basically are an interruption to what you are
> trying to actually accomplish. Simplifying them by dropping unneeded
> clutter (i.e. instant apply, removing extra buttons) is good. Making
> people stop and think about how the dialog box works is very bad. I
> doubt that most people use Gnome programs and nothing else - most of us
> work with a more heterogeneous set of applications. If 70% of them use
> one type of layout scheme, and 30% use another, it will be an ongoing
> distraction to which people will not really be able to adjust (ever use
> the trial version of WinZip?).

I believe context is strong enough that this is not a problem. You
clearly do not. I think that all in all they will have to stop and think
less, you think they will have to stop and think more.

-Seth