7 Best Practices for Better Surveys
Article by Patrick Fiorenza, RMS Director of Research Analytics –
A research study is like playing Jenga® – all the pieces are connected and if you make one mistake, the entire study will fall apart. After you’ve put in the effort to properly sample, your next challenge is to craft survey questions. This means you want to make sure that your questions are not loaded, leading, or biased, and are worded accurately to collect the data that you want.
-
Avoid leading or loaded questions.
An example of a leading question is something like, “Don’t you think that your boss is always supportive?” Rewriting the question without any assumptions and with a neutral tone will solicit a better response. A re-write might look like: “How would you rate the level of support your boss provides.”
-
Put your demographic questions last (if not needed for screening).
Placing your demographic questions at the end minimizes bias and increases the likelihood of a participant completing the survey. However, that is not always possible if you have very specific screening or qualification questions. For example, if you’re screening for age, do not wait until the very end of the survey to ask for age range, as this will clearly frustrate a participant who has already completed the majority of the questions only to be disqualified.
-
Don’t tax your participants with difficult questions all at once.
Placing questions in a row that are sensitive or require a lot of mental effort may lead to survey fatigue, among other risks (i.e., bias, decreased accuracy or negative participant experiences). By avoiding unnecessary questions and spacing out longer, more difficult ones, you will improve the quality of the data received.
-
Use the right question types.
While designing the survey, think critically about the type of question you are asking (open-ended, dichotomous, matrix, Likert, multiple choice, etc.). Each question type is suited for a particular objective, and you may need to use certain question types to get the analysis desired.
-
Avoid double-barreled questions.
A double-barreled question may look like: “How would you rate your overall satisfaction and quality of service?”. In this case, the question is asking the participant to rate two things: satisfaction and quality of service. Avoiding double-barreled questions is a good way to prevent ambiguity in responses and makes communicating findings much easier.
-
Make sure your scales are balanced.
Balanced scales are important to avoid biased responses. For this example question, “To what extent do you agree with the statement, My boss is supportive?”, an unbalanced scale may look like:
- Strongly agree
- Agree
- Somewhat agree
- Neither agree nor disagree
- Strongly disagree
Here, the scale has more positive than negative choices, making it unbalanced. Balancing the scale would add “Disagree” and “Somewhat disagree” so there is the same number of agree/disagree options on the scale.
-
Rotate/randomize non-ordinal questions.
Randomizing responses is a way to prevent response order bias, where people tend to select the first or last option regardless of content. You can make the data more reliable by simply rotating questions. However, you should only do this with non-ordinal questions, or questions where the order of the response has no ranking or important sequence. For example, you do not want to randomize a scale about agreement, as the order matters. But for something like a ranking of preferences, randomization is more appropriate.
Survey design is a mix of art and science, and these quick tips are just a few of the ways you can collect better data from your studies.
About the Author – Patrick Fiorenza
Patrick is passionate about helping clients achieve their goals and designing effective strategies through customized market research projects. He has extensive experience in survey design, data analysis, and qualitative, quantitative, and mixed-methods research. As the Director of Research Analytics, Patrick monitors all aspects of the research project, assuring validated and high-quality results are provided to RMS clients. He is also passionate about providing coaching to analytic staff and collaborating closely with clients to ensure needs are met. Pat holds a master’s in public administration from Syracuse University and is pursuing his doctorate in curriculum, instruction, and science of learning from the University at Buffalo.
It’s good to know. We’re here for you.
Call us today, to see how we can help your organization.
Interested in conducting a market research project with RMS?
Contact our Vice President of Corporate Development, Sandy Baker at SandyB@RMSresults.com or call 1-866-567-5422.
About RMS
Research & Marketing Strategies, Inc. (RMS) is a full-service market research firm in Central New York. Formed in 2002, RMS helps organizations that are looking to know more about their customers and/or potential customers. We conduct surveys, focus groups, mystery shopping, studies, and analyses. Each project is customized and gets personal attention so that actionable, data-driven findings are delivered. RMS has a reputation for getting results. We offer an independent, professional means to conduct telephone, on-line, and mail surveying, as well as in-depth interviews, intercept interviews, and participant recruitment. We also host discussion groups through QualiSight, our onsite call center and focus group facility. We have a proven reputation for successfully recruiting and moderating focus groups, community forums, and town meetings.