What if I could tell you the general consensus of how respondents will answer questions in your survey, before you even fielded it? It may sound crazy, but it’s true. At times, the way a survey question is written has more impact on the data than the respondent’s opinion on the topic. You wouldn’t think a slight wording change could have a significant impact on your data, but it absolutely can. This is especially true when surveys are written in-house, or written by someone with little to no survey design experience.
The best way to illustrate writer’s bias on survey design is to show a few theoretical examples and explain how the minor subtleties can have a major impact on data collection. Here are three quick examples:
1. The Question Should Not Lead the Respondent
Example: “How satisfied are you with the large variety of interactive activities offered at the new community center?” The leading portion of this question can be seen with the positive verbiage such as “large variety”, “interactive”, and “new”. This will result in more positive responses because the survey is already using the benefits of the community center in the question.
2. The Question Should Not Be Double-Barreled
Example: “How satisfied were you with the customer service and the wait time?” Okay, so what if the respondent has a different opinion on customer service versus wait time? This question’s setup forces the respondent to treat them the same. This is one of the more common mistakes in survey design – make sure you don’t make this costly error.
3. The Question Should Not Use Words that Overstate the Condition
Example: “How important is it to have password access to your account, which will prevent identity theft and secure your account using a valid login after a confirmation note is sent to your email?” Get to the point here and stop. If a survey has this much language you are only going to confuse the respondent. The key topic here is importance of the “password access.” Most people understand the benefit of it, so there is no need to state that, or state the process of how to set one up.
Do your surveys fall victim to these common biases? Contact Research & Marketing Strategies (RMS) – a market research firm based in Syracuse, NY to consult with you and your team on your market research needs.
Excerpts taken from Basic Marketing Research 2nd Edition, Alvin C. Burns, Ronald F. Bush.