Re: GNOME user survey 2011

[Christophe Fergeau <teuf gnome org>]

2011/8/1 Felipe Contreras <felipe contreras gmail com>:
> On Mon, Aug 1, 2011 at 1:23 PM, Christophe Fergeau <teuf gnome org> wrote:
>> 2011/8/1 Felipe Contreras <felipe contreras gmail com>:
>>>
>>> From my understanding, no number has a statistical power of 0, any
>>> number will have statistical power greater than that.
>>
>> See, you start implying that whatever we do, the numbers will have
>> statistical significance (aka science aka hard facts).
>
> I'm not implying that, that's a fact. Blame the rules of the universe
> if you don't like that.

Please note that I said "whatever we do", and in the rest of your
email you (finally ?) mention
"All this of course, depends on an unbiased sample size, which is why
as Olav pointed out, we need to identify the bias. If we properly
identify the geek bias, then we can make the calculations based only
on "non geeks"."

This was my point all along, you can't throw a survey at a random
number of people and let whoever wants answer, and expect the results
to be directly relevant. You need to do some (non trivial imo)
preparatory work if you want things to be useful in the end.

> That's is not true. While we don't know the number of GNOME users, you
> can make educated guesses about the total number of linux desktop
> users:
> http://counter.li.org/ (30 million)

Let's make educated guesses about a figure that is already an educated guess...


>
> Then, there are formulas to calculate the confidence based on the
> sample size and total population. The bigger the total population, the
> less confidence, however, after a certain point it doesn't matter much
> if the population is 10 million, or 30 million, so we can pick the
> safest one, which is the bigger one; 30 million.
>
> You would be surprised of the small sample size needed to generate a
> high level of confidence. Again, blame the universe for it's rules,
> not me.
>
> All this of course, depends on an unbiased sample size, which is why
> as Olav pointed out, we need to identify the bias. If we properly
> identify the geek bias, then we can make the calculations based only
> on "non geeks".
>
> There's plenty of things that can be done to analyze the numbers. They
> are most certainly not totally random.

Well, you summed up what has to be done to get a figure that is
vaguely meaningful to my eyes. However, I'm pessimistic that we'll
manage to derive useful data from a survey.

Christophe