Re: GNOME user survey 2011

[Felipe Contreras <felipe contreras gmail com>]

On Mon, Aug 1, 2011 at 11:31 AM, Christophe Fergeau <teuf gnome org> wrote:
> 2011/8/1 Felipe Contreras <felipe contreras gmail com>:
>> I other words, you are saying that it doesn't matter if 100% of the
>> responders of this survey say GNOME has too few options, nothing would
>> be done? Is there *any* kind of evidence that would convince GNOME ppl
>> that users want more options? Or is it what the wishes of users are
>> completely irrelevant?
>
> It seems you are starting under the assumption that the results of the
> survey will be representative of G3 users as a whole.

My assumption is that some numbers are better than no numbers.

> What are your
> plans to make sure people unhappy with GNOME are not over represented
> in the poll results?

Publicize the survey as much as possible.

Would you like to rephrase the survey to don't assume the respondent
is using GNOME, and then ask this question?
== Which desktop environment are you currently using? ==

This should spot the people that are more likely to be unhappy.

> Given the kind of questions, it's bound to
> attract answers from people who want more options, and I don't think
> how we can go from "N% of the people who took the survey said they
> wanted more options" to "M% of *all* G3 users want more options". To
> me, these figures will be totally unrelated, unless I missed something
> in the way you want to run the poll.

They are certainly not the same, but if N = 100, you can say with
certain degree of certainty that M is certainly not 0. Which value of
N will convince you that M is high enough, I don't know (perhaps there
isn't any). But trying to get a value of N as close as M as possible
is better to what is available now, which is nothing.

> Since your actual goal seems to use this survey results to pressure
> people into adding more options to GNOME (where "more options"
> probably really means "the options you want to have"), I'm afraid this
> poll will turn into "I want to prove X, let's design a survey whose
> result will be X".

That's speculation. How would you design a survey to prove !X? It will
be the same.

> I even suspect that you'd get different results by
> adding something like "Do you get confused by software with too many
> configuration options ? Yes/No" " before asking the question about the
> amount of configuration options ;)

So you want to add bias? If you have been following the thread, you
would see that the proposed changes are intended to decrease the bias.

> In short, I think making a good poll is really hard, especially if you
> want to use this poll to prove that some specific point is true or
> not. Imo, the best we can get from a poll is "ok, some people think X,
> unfortunately we have no idea about what the people who did not answer
> the poll think"

So you prefer, the status quo, which is "we have no idea about what
anybody thinks".

I say the reasonable thing to do is to try to improve the survey as
much as possible to get as meaningful numbers as possible. After
getting the numbers, _then_ you can decide what to do with them, if
anything at all.

-- 
Felipe Contreras