I’m hoping that I have a few helpful academics amongst my readers here.
As I guess most people know, a lot of research in various fields revolves around questionnaires. You get this in politics, in marketing, and in sociology. The matter of questionnaire design is therefore important, because badly designed questions can bias the results, right?
Now suppose you are designing a survey to measure public attitudes towards something, say science fiction. It seems to me self-evident that if your questionnaire is relentlessly negative about the subject then you will a) encourage a negative response and b) leave your respondents with a more negative view of the subject than when they started. To illustrate the point, here are two short sets of questions.
On a scale of 1 (low) to 5 (high) please rate the following:
1. How intelligent are science fiction readers compared to readers in general?
2. What is the quality of writing like in science fiction compared to other fiction?
3. How likely are you to want a science fiction reader as a friend?
Less Neutral Questions
One a scale of 1 (strongly disagree) to 5 (strongly agree) please indicate how you feel about these statements:
1. Science fiction readers are stupid people who live in fantasy world.
2. Science fiction books are very poorly written.
3. Science fiction readers are dull, boring people with poor personal hygiene.
See what I mean?
I’m sure that someone, somewhere has done research into the comparative effectiveness of these two strategies. But finding that research isn’t easy, and an academic who has inherited the “less neutral” methodology from previous work in the field isn’t going to be able to challenge those previous methods without proof that they are suspect. Has anyone out there ever done any work on questionnaire design, or read any work on it, that might help?
Of course the survey I’m actually interested in isn’t about science fiction readers, but I think you can probably guess what sort of social minority it is still viewed as acceptable to study in this way.