Re: GNOME user survey 2011

[Olav Vitters <olav vitters nl>]

On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters <olav vitters nl> wrote:
> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
> >> Many of these are borrowed from the Git user survey. The results as
> >> you can see, can be quite interesting:
> >> https://git.wiki.kernel.org/index.php/GitSurvey2010
> >>
> >> It would be great if some sort of notification would popup directly on
> >> user's desktops, this way it can ensured that the maximum amount of
> >> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
> >
> > That's impossible.
> 
> Everything is possible.
> 
> >> Google+ and so on should give plenty of feedback. Maybe also contact
> >> Ars Technica, LWN, Phornix, and so on would help.
> >
> > Those are only technical sites. I think the results are going to be
> > biased whatever you do.
> 
> How do you suggest to reach the end users if you already dismissed a
> pop up directly from the desktop? If there isn't any way, at least
> it's better than nothing.

I didn't dismiss. I said it was impossible to popup a notification.

Survey is only good if either the bias can be determined, or if it is
unbiased. As mentioned, I think it will be biased, so the survey should
have some questions to determine the bias.

> >> === 03. How do you describe the amount of configurations available? ===
> >
> > I don't see the relevance of asking this. Furthermore the question is
> > suggestive. Seems more to prove a point than anything else.
> 
> I do see the relevance, as I think it has been a big point of
> contention raised by many users.

Something should be done with a survey. No matter the outcome of this
question, you won't be able to take these results and change things.

Asking if people want more configuration options goes against why
options are removed. Ideally everything should happen automatically.

I'm only interested in the cases where it doesn't work.

> If you don't think it's very important, it could go to the end.
> Besides, if the results are overwhelmingly pointing to "Too few"?
> Don't you think it would be worth investigating? If the results end up
> being "Just enough", then no harm done.

The question is suggestive so the results will be biased. It is better
to ask something like "does GNOME do what you want?" and then have a
text field where they can specify what GNOME lacks.

> >> === 04. Which GNOME version(s) are you using? ===
> >
> > This should be asked earlier.
> 
> Where?

before #3.

IMO the questions should be ordered so that the basic/factual stuff is
first, possibly ask what they like, then questions where you just say
what you feel, and every improvement suggestion should come last.

> >> === 05. How do you compare the current GNOME version with the version
> >> from one year ago? ===
> >
> > This is vague. Current as in GNOME 3.2 / 3.0, or current as in the
> > version that they are using. Furthermore, you don't know if they upgrade
> > each year.
> 
> Obviously it's the one they are using, otherwise how would they compare?

That should be made clear in the question, otherwise it will be
interpreted differently and the results are biased.

> Would this help?
> How do you compare your current GNOME version with the version from
> one year ago?

Like it.

> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
> >
> > With what purpose is this asked? No support is given on GNOME Bugzilla.
> > Only minimal on gnome-list.
> 
> It's important to have a two-way communication with the users, don't
> you think? So it's important to figure out what channels they actually
> use. In fact, I was thinking to add another question asking if they
> feel there's lack of communication with the team.

I don't see giving support as:
 * something which shouldn't be done
 * two-way communication
 * a way that contributors communicate amongst each other

However, we don't really give support. It would be nice to do more,
but.. it is not done that much (bit on IRC, mailing lists, some forums).
Though I'll say beforehand that I see support as separate from a
developer task.

> Resolving bugs is also support. But feel free to rephrase the question
> to find out how users provide any kind feedback; questions, bug
> reports, issues, etc.

I again disagree. A support issue can be caused by a bug. Such a bug can
be a requirement to solve the support issue. But this doesn't mean
fixing bugs is support.

I still do not see the relevance of this question. Perhaps we could ask
where they learned of the survey?

> >> === 08. Do you have any comments or suggestions for the GNOME team? ===
> >
> > How will you handle all the various comments you'll get? Git received
> > 9000 answers.. who is going to read them all and summarize?
> 
> I would, if nobody takes the task. Most likely I will miss a lot of
> important stuff if I'm the only one. But it is important to ask this,
> as the survey is bound to be imperfect, and this is the only place
> where people can actually suggest what to add to the next survey
> (indirectly).

Nice.

Note: I do like the question, my only interest is that we don't ask
something, then ignore the answers.

> > Things I'd like to know:
> >  - Multiple choice options to understand the responders technical
> >   skills. So if they've submitted a patch, contributed code somewhere,
> >   paid to write software, maintain some open source code, etc.
> >   That is IMO a nicer way to understand their skills can someone to
> >   rate themselves.
> 
> How about Zeeshan's suggestion of asking how often they use the terminal?
> 
> >  - If they use KDE/XFCE/something
> 
> That's confusing. You mean if they switch between GNOME and those? Or
> that they used to use GNOME, and now they use that one?

My thought was if they use multiple at the same time.

> How about:
> Which other desktop environments have you used?

I like that. Maybe add a timeframe? E.g. "used in the last three years"?

> >  - other things to later on be able to determine if the survey is biased
> >   in some way. Don't have concrete ideas atm.
> 
> Well, how are you going to determine if the survey is biased in some
> way, or it's the actual GNOME users that are biased in some way?
> Either way, I think the only important bias is geekness.

Don't know, that is why I said I don't have concrete ideas.
-- 
Regards,
Olav