On Fri, 2011-08-19 at 16:08 +0300, Felipe Contreras wrote: > >>> Likewise, > >>> 'happy' will be thought of differently by different people (a very odd > >>> word to include in a questionnaire, if you don't mind me saying): > >> > >> I think everyone understands the word happy. > > > > /ME wipes a mouthful of coffee from my monitor > > > > Then you haven't read enough of the survey research literature. > > That doesn't change the fact that everyone understands the word "happy". > Not necessary. Just to give an example - there is strong cultural influence how do you respond to simple question 'How are you'. In some cultures it is impolite to answer better then 'so so' and the normal answer is somehow along lines 'it could be worst, it could be better'. On the other hand the correct response in English is usually 'great' or 'fine' (to quote my teacher 'even if your house is burnt and your dog is terminally ill'). I have been warned to avoid 'standard' 'so so' response as I will be perceived as either impolite or after some large disaster because what I really meant was 'great'. (Somehow less directly related but also illustrates the problem of tricky words - in my native language friend means what in English is understood by close friend and many people whom I would call in English friend I would call in Polish acquaintance. Even though I know the difference I am less inclined to call people friends as in my mental model they are described by word 'acquaintance'). Of course this is 'just' cultural bias caused by people not being native speakers of English. You need to add individual bias. In each case it adds more and more 'noise' to survey. > >>> tl;dr version: your survey results will be misleading. > >> > >> No, the results would not be misleading; the *analysis* of the results > >> might. But different people can analyze them in different ways. The > >> important thing is to get *some* results. > > > > It seems bizarre to suggest that research data is valid irrespective > > of how it is gathered. If your questionnaire does not provide valid > > measurements no amount of analysis can compensate. > > You can thrown an analysis saying all this data is crap if that makes > you happier, but this survey won't eat babies. > I would argue that incorrect data (misinformation) is worst case then no data at all. Regards
Attachment:
signature.asc
Description: This is a digitally signed message part