What if I could tell you the general consensus of how respondents will answer questions in your survey, before you even fielded it? It may sound crazy, but it’s true. At times, the way a survey question is written has more impact on the data than the respondent’s opinion on the topic. You wouldn’t think a slight wording change could have a significant impact on your data, but it absolutely can. This is especially true when surveys are written in-house, or written by someone with little to no survey design experience.
The best way to illustrate writer’s bias on survey design is to show a few theoretical examples and explain how the minor subtleties can have a major impact on data collection. Here are three quick examples:
1. The Question Should Not Lead the Respondent
Example: “How satisfied are you with the large variety of interactive activities offered at the new community center?” The leading portion of this question can be seen with the positive verbiage such as “large variety”, “interactive”, and “new”. This will result in more positive responses because the survey is already using the benefits of the community center in the question.
2. The Question Should Not Be Double-Barreled
Example: “How satisfied were you with the customer service and the wait time?” Okay, so what if the respondent has a different opinion on customer service versus wait time? This question’s setup forces the respondent to treat them the same. This is one of the more common mistakes in survey design – make sure you don’t make this costly error.
Consequences of Poor Survey Writing
3. The Question Should Not Use Words that Overstate the Condition
Example: “How important is it to have password access to your account, which will prevent identity theft and secure your account using a valid login after a confirmation note is sent to your email?” Get to the point here and stop. If a survey has this much language you are only going to confuse the respondent. The key topic here is importance of the “password access.” Most people understand the benefit of it, so there is no need to state that, or state the process of how to set one up.
Do your surveys fall victim to these common biases? Contact Research & Marketing Strategies (RMS) – a market research firm based in Syracuse, NY to consult with you and your team on your market research needs.
Excerpts taken from Basic Marketing Research 2nd Edition, Alvin C. Burns, Ronald F. Bush.
I work for a rhinoplasty surgeon in New York and we recently talked about doing client satisfaction surveys. Right now we’re weighing the possibilities of doing it in house rather than hiring an outside service. These are some great tips to help us in our decision.
Glad this helped Noelle, I encourage you to check back for more survey tips. Feel free to fill out our quick readership survey (if you haven’t done so already – https://rmsresults.com/survey/blog_readership_survey/) to give us some suggestions on what you’d like to read and enter for a chance to win a $25 gift card. Let us know if we can be of any further assistance.
Here is another good post on the “in-house” versus “third party” topic our Senior Analyst wrote: http://wp.me/pTbw8-3B
[…] in the Bunker, we take great care to avoid crafting survey questions that might bias respondents or influence the responses in a certain direction. It can be quite a challenge and sometimes requires a great deal of subtle wordsmithing. It isn’t […]
[…] people know that the wording of a question on a survey can have an impact on the way respondents answer it. A less obvious, but potentially […]
[…] Check your question wording – Oftentimes when someone is writing their own survey, they will have an objective in mind or even a strong belief about the survey topic. This will cause them to unintentionally bias the data based on question wording. An extreme example would be a question like this: Do you agree with everybody else that my business is the best around? Other things to watch out for are double-barreled questions and overstating a condition. See our blog post here for examples. […]