Pierre-Auguste Renoir: The Ball at the Moulin de la Galette (1876).



Poorly worded questions affect the content validity of survey findings. Unfortunately, it is easy to make mistakes when writing items for a survey instrument. For mailed surveys, poorly worded questions are particularly bothersome because mistakes cannot be corrected after the questionnaires are mailed.

The Babbie textbook offers suggestions for writing good questions. Suggestions can be found also on Pp. 140-143 in Delbert C. Miller, Handbook of Research Design and Social Measurement, Fifth Edition. The list presented below comes from Survey Research by Telephone, Second Edition by James Frey, with some examples of poorly worded questions provided by Dr. Sapp.

Examples of Common Wording Problems When Writing Questionnaires:

1. Avoid the loaded question:
   
   Do you think persons should be able to smoke cigarettes anywhere they please, regardless of how they affect the health of others?
   
   The Surgeon General states that cigarette smoking is harmful to one's health. Do you encourage your children to smoke cigarettes?
   
   Note: Sometimes the researcher might deliberately want to use biased wording to help balance a controversial topic: Do you support cigarette advertising in foreign countries to promote job creation in the U.S.?
   
   
2. Avoid the use of inflammatory words:
   
   Do you think rude people should be able to smoke their cigarettes while attending a baseball game?
   
   
3. Avoid being too folksy:
   
   Ok, let's look at some questions on smoking cigarettes.
   
   
4. Avoid slang terms:
   
   Would you hang with a smoker?
   
   
5. Avoid technical terms:
   
   Approximately how many PCP's are inhaled from smoking one cigarette?
   
   
6. Use precise concepts:
   
   Should tobacco be banned?
   
   
7. Be precise regarding time:
   
   Have you ever smoked cigarettes (meaning "as a habit" rather than "ever tried one")
   
   
8. Use accurate facts:
   
   How concerned are you about the possibility of contracting AIDS from cigarette smoking?
   
   
9. Do not assume knowledge or behavior:
   
   Do you agree with the Surgeon General's latest report on cigarette smoking?
   
   
10. Use correct grammar:
    
    Should cigarette smoking be gotten done with?
    
    
11. Avoid double negatives:
    
    Do you disagree that cigarette smoking is disagreeable?
    
    
12. Avoid the double-barreled question:
    
    Do you think smoking cigarettes is bad for your health and well-being?
    
    
13. Response categories should match the question:
    
    Should the national health care bill include a $1.00 tax increase on a pack of cigarettes?
    
    1. never
    2. sometimes
    3. often
    4. always
    
    
14. Response categories must be mutually exclusive:
    
    How much do you spend on cigarettes each week?
    
    1. Do Not Smoke
    2. less than $10
    3. $10 to $15
    4. $15 or more
    
    
15. Use a time frame to measure future behavior:
    
    Wrong: Will you ever smoke a cigarette?
    Right: Do you intend to smoke a cigarette within the next week?
    
    
16. Avoid determinism:
    
    Is cigarette smoking in public places ever acceptable?
    
    
17. Include "don't know" only when appropriate:
    
    Dr. Sapp advises to use a "don't know" response category when requesting factual information (e.g., Do your children smoke cigarettes?), but not when requesting opinions (e.g., Should billboard advertisements for cigarettes be banned?).
    
    
18. Split complex questions into parts:
    
    Wrong: What percentage of your weekly income do you spend on cigarettes?
    
    Right: What was your total income before taxes in 2007?
    Approximately how much money do you spend on cigarettes each week?
    
    
19. Provide clear instructions on responses:
    
    Please rate your opinion of smoking cigarettes on a scale from 1 to 10.
    
    
20. Avoid specifying response alternatives in the question:
    
    Do you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree that cigarette smoking is harmful to one's health?
    
    
21. Avoid lists longer than five items:
    
    Please rank in order of importance the following 15 reasons for avoiding cigarette smoking.
    
    
22. Avoid too much abstraction:
    
    Does cigarette smoking erode the moral integrity of the American citizenry?
    
    
23. Be simple without being condescending:
    
    Should the Surgeon General (i.e., the head person in charge of health promotion) ban cigarette smoking?

Other Notes:

Avoid lengthy questions.

Special instructions to interviewers should be clear and easy to follow.