As previously defined in one of the Bunker’s prior post, survey coding is the process of taking the open-end responses and categorizing them into groups.  Once coded, they can be analyzed in the same way single and multiple response questions can be.  So, the process is fairly simple.  It usually involves looking at exported data on an Excel sheet, adding columns for coding, and reading through the 0pen-ends and assigning each open-end with a category code.  From there you can subtotal the codes and divide them by your pool to assign representative percentages.

What is survey coding in market research?  Here is an example:

The process seems very simple, but there are many factors that contribute to coding that make it unscientific.  Research & Marketing Strategies (RMS) – a market research company in Syracuse, NY – utilizes coding in almost every single quantitative report sent to clients.  It’s extremely valuable for clients (and much easier) for them to be able to see a percentage breakdown of how respondents answered a question rather than having them read through 800 open-ends.  However, caution should be taken when reviewing any coded question.  Here are a few reasons survey coding is not a scientific and fool-proof process:

1) Survey coding is subjective – if you were to take three separate analysts and have them code the same identical 100 open-ends, you will undoubtedly end up with three very different graphs.  Not only will you end up with different percentages for each category but most likely different categories as well. You could argue that even the same analysts would code the data differently depending on the day.  It helps to create some kind of consistency with codes, either referring to a past project with that client, or using the same analyst to code questions from the survey.  It also helps if you know if there are any specific topics that the client would like pulled from the open-ends and you can be sure to create a category for that.  For example, if a client wants to find out how many people referenced the “Christmas special” on the question about content of advertising, you can ensure that a category in your graph will depict that.

2) Processes differ – some analysts will limit coding to using just one all-encompassing code, while others could decide to use up to four or five if the open-end covers a wide array of topics.  On top of the decision to use codes per open-end, some analysts will aggregate the statistics by percentage of responses while others will use percentage of respondents.  We endorse the percentage of respondents.

Now, I am not arguing that you should not use survey coding in a report.  In fact, for previously stated reasons, it offers analysts and clients both major benefits that I believe outweigh the subjectivity of the process.  Other substitutes for coding of open-ends that seem to be a nice supplement are using Wordles in our PowerPoint reports.  These Wordles encompass all words used in verbatim responses and are able to highlight the words used most often by respondents in larger font.  A second option is limiting the scope of the open-ended question to force the respondent to eliminate interpretation from the analyst.  Instead of asking “when you think of client XYZ, what comes to mind?,” you may want to ask “when you think of client XYZ, give me one word or phrase that comes to mind.”  By limiting the scope of the response, it eliminates additional details and cuts right to the core of the question (and it will save you hours of coding – You’re welcome!)

Do you have questions about coding or how surveys can benefit your business?  Contact our Business Development Director Sandy Baker at SandyB@RMSresults.com or by calling us at 315-635-9802.