It goes without saying – some level of pre-testing is a must before launching a full-scale survey.
Survey Sampling International (SSI) defines a pre-test as a questionnaire that is tested on a statistically small sample of respondents before a full-scale study. The benefits of pre-testing cannot be overstated whether you go to the extent of a soft launch or not. The real issue is, in an industry where time is often of the essence and findings need to be turned around yesterday, it’s difficult to budget the time into a study to conduct a thorough pre-test. In some cases, one could argue a pre-test is not necessary, such as a renewal of a yearly survey, or an online survey that is only being sent to a small population. Here are some reasons as to why pre-testing your online survey is vital. Keep in mind, some of these benefits will also apply to other methodologies including phone surveys.
1. Estimating a response rate.
In all likelihood, this is one of the most common reasons surveys are pre-tested. When you send 1,000 survey invitations out and obtain 35 completes, you know that your response rate overall will hover around 3.5%. This helps you guide future sample purchase and determine how many invites you need to send to reach your quotas, without having to overspend.
2. Fishing out wording issues or questions that are difficult to answer.
As survey writers, we’ve all been in situations where we spend so much time engulfed in a survey that we overlook or don’t catch some minor issues. By pre-testing your survey, you are getting a fresh perspective on the question language. Doing this on a smaller scale, such as an email to other colleagues can accomplish the same results.
3. Estimating the length of time to complete the survey.
This factor is probably the second of the two most common reasons surveys are pre-tested. By reviewing the time stamps of how long it took the respondent(s) to take the survey you can determine whether you can add those two other key questions your client wanted to add, or determine if you need to cut back on the scope.
4. Understanding points of likely dropout.
We’ve often spoken about better engaging survey respondents in past blog posts and this benefit speaks directly to that. You can accomplish this a few separate ways. One would be to look at data to see which questions carried the highest likelihood of dropping out. Another, more personalized method would be to follow-up with select survey respondents by phone to discuss their survey experience (if you have access to their telephone number through an internal or client panel).
5. Improving your subject line or invite text.
Another benefit of a pre-test is to test the effectiveness of your subject line or invite text. Spam filters are a real concern when it comes to online surveys. Therefore, if you have time to pre-test, you might want to think about using a few different subject lines or invite text and then see which of those worked best.
Are you thinking about conducting an online survey for your business? Contact Sandy Baker at 315-635-9802 or SandyB@RMSresults.com.
Nice article. At InSites (www.insites.eu) we involve our members in this. We have a group of panelists that pre-test our surveys within 24h sending their feedback immediatly to the project team before launch. We ask them to check on language (as we work with a translation agency), questioning, technical performance and overall feedback. Works fine! And not that hard to implement. We call them our Watsons team 🙂
[…] findings, and quality recommendations. In previous blog posts we’ve talked about things such as pre-testing, call-center quality vs. quantity, and things to look for when proofing your market research […]
[…] of testing your online survey before launch, which we have covered before here in the Bunker with this post. The article offers six quality control checks before launching your online survey using the […]
Nice article. Everybody who is interested in more detail of our product SurveyTester can contact be at bwitt@2×4.de . SurveyTester allows structured feedback, programmers can do automatic re-tests and the management team knows exactly how many tests have been done, how many issues were found, which of them were fixed, etc. A basic description is also available at http://www.2x4technology.com/de/SurveyTester/SurveyTester.aspx.
[…] quantity inputs in survey software, but you can be sure he or she would have taken plenty of time pre-testing a survey like this before it was launched. Also, the sender should know you only get one chance with market […]