Re: [Rhythmbox-devel] Usability report on Rhythmbox

[Charline <charline poirier canonical com>]

On 29/07/10 14:51, Jonathan Matthew wrote:

Hi Charline,

On Fri, Jul 23, 2010 at 12:38 AM, Charline
<charline poirier canonical com> wrote:
  

Hello,

I have just completed usability testing of Rhythmbox.  I am planning to
publish the results on CanonicalDesign.com on Tuesday.  I would like you to
read the report, if you have time, and send  me feedback or any questions
you might have.

I hope the report will be helpful.
    

First of all, many thanks for doing this testing. We don't really have
the capacity to do this ourselves, so the only feedback we get is from
users who are motivated and educated enough to report bugs, post to
the mailing list, or talk to us on IRC, which naturally skews towards
more experienced and technically capable users.

The main messages I'm seeing here are that we need to put more effort
into explaining how Rhythmbox interacts with various music sources -
what happens when you import files, what that means when they're on a
removable device; how Rhythmbox treats podcasts and MP3 player devices
and so on; that we need to highlight actions other than playback,
since users are more likely to want to extract an audio CD into their
library than to play from it, and similarly with other devices; and
that we should reduce the time it takes for first time users to make
Rhythmbox do something, by providing shortcuts for importing music
from various locations (somewhere on the filesystem, removable
devices, network shares, etc.). 

Yes, I think that some dialogue would greatly improve the experience and would make users more tolerant of glitches and bugs.

I have some plans for some aspects of
this, but I'm not sure when I'll be able to work on any of it.

My feedback on the report itself:

If UX advocates or developers on the project are part of the intended
audience, it'd be really helpful to clarify the terms you use for
actions and parts of the interface with them ahead of time. I'm having
trouble understanding the descriptions of user actions and
expectations and mapping them to specific parts of the Rhythmbox
interface, and at times it seems like what's being described isn't
part of Rhythmbox at all.
  

You are right.  I was thinking of adding screenshots - do you think that a visual would suffice?

Again, if people who work on the project are part of the intended
audience, starting with 3 and a half pages of unrelenting criticism
isn't a great idea. 

I have been worried about this.  It is difficult to communicate that a usability report is not a criticism.  If users are challenged using an application, it doesn't mean someone messed up, it only means that the communication between the app and its intended users is not optimal.  We all do our best to try predict our users but they are complex, they have many goals,  and so very surprising... Often, the serious problems are caused by small oversights, like with Rhythmbox, participants felt out of control because Rhythmbox did not communicate what it was doing. A small thing in comparison to all the work that was required to plan, build and deploy it.  In addition, it is not a criticism because it is a simple description of what I observed.  Personally, I have no value judgment about it.  I think that usability testing can help you make decisions.  You see the evidence and you make your own choices knowing what the consequences can be.  

These are the only people in a position to fix the
problems you're finding, and if you put them on the defensive, they'll
mostly ignore your report, or if you're especially lucky, blame the
conditions of the testing or even the users for everything that went
wrong. I certainly found myself tempted to do that.
  

Glad you didn't :-)

Where you make suggestions for simple improvements, or identify
specific missing features, I'd encourage you to file bugs in the
project bug tracker. For example, from this report, your suggestions
about changing the label for the CD extract button and adding
explanations for the 'missing files' and 'import errors' sources could
be cut and pasted straight into a bug report.
  

Will do.

I'd like to see more information on how the tasks were presented to
the users. Particularly with the podcast task, it seemed as though
you'd instructed users unfamiliar with podcasts to download one, which
seems to me to be the opposite of how users come to find the podcast
feature in Rhythmbox - they find some media that comes in podcast
form, then search for an application that handles podcasts. 

I will make sure to clarify this in the report.  Concerning the podcast task:  participants were familiar with podcasts, they were podcast listeners.  I asked them to download their favourite podcast into Rhythmbox.

In some of
the other tasks, I'm not sure what the user was actually trying to
achieve, which, when also I can't comprehend the descriptions of what
they tried to do, means I'm at a loss to understand how we can improve
the situation.
  

I will add a description of the tasks in the report.

Again, thanks for doing this, and I hope we can turn this report into
some useful improvements, not just for new users but for everybody.
  

Thanks, Jonathan, for taking the time to send me this feedback.  It will greatly improve my report.

-jonathan
  

--

CHARLINE POIRIER
User Research Programme Lead
Canonical
27th floor, 21-24 Millbank Tower
London SW1P 4QP UK 

Tel: +44 (0) 20 7630 2491
Mob: +44 (0) 78 8695 4514
www.Ubuntu.com <http://www.Ubuntu.com/>
www.Canonical.com <http://www.Canonical.com/>