As previously cited in this blog, online surveys have become the most common survey mode in market research – true for Syracuse, Central New York and beyond.  Online surveys are done quicker and cheaper than the more traditional mail or phone methodologies, thus clients are choosing that mode instead of others.  In the process, be careful to not lose your focus on data integrity as your clients request online surveys. 

There are 3 basic types of online survey offenders the RMS research bunker team watches for:

1. “The Daydreamer” – this respondent does not read the questions or stay focused enough to understand what is being asked.  Known Accomplices: “The Woolgatherer,” “The Aloof,” “Mr. or Ms. Elsewhere”

2. “The Imposter” – this respondent provides false data to a) bias survey results or b) fraud data to qualify for a survey with an incentive being offered.  Known Accomplices: “The Survey Swindler,” “The Con,” “The Pretender”

3. “The Speeder” – this respondent races through the survey in record time, click after click after click, leaving a trail of data integrity destruction in his/her path.  Known Accomplices: “The Express,” “The Research Runner,” “The Straightliner”

How do you stop them?  Here are five suggestions:

  • Suggestion #1: Find a trustworthy sample house.  A lot of the imposters can be eliminated through filtering of sample.
  • Suggestion #2: Add a hidden variable for TTC (time to complete) the survey.  Set a parameter during the data collection process.  Plan to remove the first 2% or 5% of outliers who completed the survey in the least amount of time.  Or you can review the TTC variable manually; if the tested survey takes 12 minutes to complete and you have respondents who completed it in under 5, you have identified your speeders.
  • Suggestion #3: Take a demographic question you asked in the beginning of the survey and repeat it at the end (age, zip code, number of people in household).  If the answers do not match, you’ve singled out the 3 offenders.
  • Suggestion #4: Work in a trap or verification question – ask the respondent to select #4 below or type ABC in the box below.   Another option is to list a product/service/company that does not exist and place it in the grid with others.   If they do not select or answer correctly or rate something that doesn’t exist, disqualify them from the survey.  Some companies use opposite wording or double negatives for this.  I argue against it as it confuses respondents and often frustrates them to a point of dropping out.

  • Suggestion #5: Simply making your surveys concise, engaging and interesting will eliminate a lot of online data integrity issues and stop these 3 imposters!

Please Note: Not all online surveys require these types of controls – many audiences will be attentive, credible and will answer the questions thoughtfully.  Just work with your market research provider to determine whether or not these quality checks need to be put in place.  This article is loosely based off a Quirks article from August 2008 titled By the Numbers: Protecting online survey data integrity by author Chris Gwinner.