One of the more common questions in market research is “How many people do we survey?” Those who have not spent a lot of time working with surveys are not sure what the difference is between collecting 100 completes in comparison to 1,000 completes. I am sure they understand the inherent concept that 1,000 completes will be more accurate than 100 completes, but they probably don’t understand the details and specifics of the accuracy. Margin of error is a daily conversation for someone working in market research.

Here are four items to consider when you are trying to determine “how many people do we survey?”

• Margin of error – most political polls require a low margin of error of +/- 2% or 3%. Depending on the confidence interval (let’s use 95% as an example), the margin of error means that if the same survey was conducted 100 more times with a random set of people, that 95 out of 100 times each statistic in the report asked of everyone would fall within %2 or 3% of the reported number. Most of our standard surveys aim for 400 completes, which is approximately a +/- 5% margin of error. This threshold seems to be a nice cross between accuracy of results and cost of completes. Margin of error is also highly dependent on the pool or population. If you collect 400 completes among an audience of 950, it will be more accurate than 400 completes among an audience of 950,000. Trying to determine the margin of error? Use our RMS margin of error calculator.
• Budget – this factor is a common driver to the number of completes in our studies. Sure, every client would love to collect 1,000 completes for their telephone surveys but the cost of fieldwork for that number is far more expensive than the standard 400 completes. We’ll often see clients request more completes especially if they want a key break or cross-tab in the data (e.g., customer vs. non-customer.) In this case, we’d recommend collecting a significant number of completes for both and in total. If 1,000+ completes is a must-have for your study, you do have other options including switching methodologies to online, which may and probably will cost less than a traditional telephone survey.
• Timeframe – this may also dictate the number of completes. If you or your business needs a quick turnaround on the study, you may have to settle for a smaller number of completes. Online surveys provide an advantage here because you can send all of your invites at once, and you’ll get a large majority of completes quickly. With a telephone survey, you’ll be slowly progressing minute-by-minute, hour-by-hour, and day-by-day. However, depending on the exact number of completes, some call centers can turnaround a large number of completes within a day or two.
• Number of people in pool – this factor is more about managing expectations when you are asking “How many people do we survey?” Response rates are often overestimated. Let’s talk about the 2010 United States Census. This Census is designed to obtain household information from people across the country and a response is “required” by law. The government spent $1.3 billion dollars on this effort and achieved a 74% response rate. So, just temper your expectations. If you have 433 people you want to survey, chances are you will get nowhere near a 100% completion rate unless it is a controlled environment. Standard telephone surveys to consumers experience a 1:10 ratio (meaning one complete for every 10 records you call after three attempts), and business-to-business surveys are more around 1:20.

So if you are wondering how many completes you need for your survey, consider those four factors and make the best choice based on your needs. Research & Marketing Strategies (RMS) is a market research firm in Upstate NY. If you are interested in consulting with us on your current or upcoming survey work, contact our Business Development Director, Sandy Baker, at SandyB@RMSresults.com or by calling 315-635-9802.