Article by Patrick Fiorenza, RMS Director of Research Analytics –
One of the challenges faced in any market research study is gaining proper representation from panels or opt-in surveys. At times, RMS has implemented post-field work adjustments through weighting techniques. The use of weighting helps the results be more representative of a population.
There are many different weighting strategies you can use, two of the most common are called “raking” and RIM (Random Iterative Method) weighting. You may also hear the term called “Iterative Proportional Fitting (IPF).” Propensity Score Matching is another advanced technique that can be applied. You may even do multiple weighting techniques if you have numerous datasets and intend to use a matching technique. For example, you might do propensity score matching and then raking.
What’s confusing is that all these terms are often used interchangeably but essentially getting at the same thing – manipulating a dataset to correct sampling errors making results representative of a population.
Benefits of Using Weighting Techniques
From our experience, below are some of the many benefits of using weighting techniques in a study:
- Enhanced representativeness: this is the primary goal of weighting—to assure that the data collected represents the intended population. Without representativeness, any conclusion drawn cannot be generalized to the population studied.
- Reduced bias: Weighting offers the ability to correct imbalances in a survey. For example, gender is a very common variable in which weighting is used. We also know that women tend to complete more online surveys than men, so a weighting technique can help reduce bias in cases where the percentage of men and women who took a survey is skewed.
- Increased statistical validity: Weighting is a strategy that can help ensure the results are what you would expect if you surveyed the total population. For example, if you were doing a study looking at healthcare in a community, you would want to be sure that the results from your survey are representative of the whole population. Weighting can help increase the statistical validity by adjusting the responses to match the total population.
- Improved trust in decision-making: Ultimately, our goal at RMS is to facilitate making better decisions. Whether the study is to improve business strategy, marketing initiatives, or analyzing any type of policy, you want trust in your data.
Although it is technically possible to use Excel for weighting, the best way is to use a more advanced statistical tool, like R, SPSS, or Stata. Some survey software’s analytical packages (including what RMS uses) can also conduct weighting techniques.
Weighting In Practice – An Example
To best understand how weighting might be used in a study, we have created a brief case study below. Let’s say that a major technology company wants to better understand consumer purchasing behaviors and preferences for a new product they are developing. They believe their target market is single individuals aged 25-35.
Partnering with RMS to design and administer the survey, RMS used their ViewPoint panel and conducted a stratified random sampling technique. The survey included various questions to screen participants to ensure they were part of the target market in the study.
RMS and the client collectively developed key survey questions about purchasing history, brand awareness, desired product features, and demand for the product. The survey also included a suite of demographic questions. The initial analysis by RMS found that the survey skewed strongly toward females.
RMS leveraged key demographic data from the US Census and created the necessary weights for gender. RMS then used their statistical software package to adjust each response based on known demographic characteristics, continuing the process until the dataset closely matched select demographic data from the Census. Proper data checks were then conducted, and RMS compared the weighted sample to the demographic data. With the enhanced data, RMS could report findings with more confidence to give the client informed recommendations.
Although weighting is a powerful technique, it is not always the right solution for a study. Important considerations such as sample size and research needs should be considered. Partnering with RMS, we can provide insights and help you determine if weighting is right for your study.
About the Author – Patrick Fiorenza
Patrick is passionate about helping clients achieve their goals and designing effective strategies through customized market research projects. He has extensive experience in survey design, data analysis, and qualitative, quantitative, and mixed-methods research. As the Director of Research Analytics, Patrick monitors all aspects of the research project, assuring validated and high-quality results are provided to RMS clients. He is also passionate about providing coaching to analytic staff and collaborating closely with clients to ensure needs are met. Pat holds a master’s in public administration from Syracuse University and is pursuing his doctorate in curriculum, instruction, and science of learning from the University at Buffalo.
It’s good to know. We’re here for you.
Call us today, to see how we can help your organization.
Interested in conducting a market research project with RMS?
Contact our Vice President of Corporate Development, Sandy Baker at SandyB@RMSresults.com or call 1-866-567-5422.
About RMS
Research & Marketing Strategies, Inc. (RMS) is a full-service market research firm in Central New York. Formed in 2002, RMS helps organizations that are looking to know more about their customers and/or potential customers. We conduct surveys, focus groups, mystery shopping, studies, and analyses. Each project is customized and gets personal attention so that actionable, data-driven findings are delivered. RMS has a reputation for getting results. We offer an independent, professional means to conduct telephone, on-line, and mail surveying, as well as in-depth interviews, intercept interviews, and participant recruitment. We also host discussion groups through QualiSight, our onsite call center and focus group facility. We have a proven reputation for successfully recruiting and moderating focus groups, community forums, and town meetings.
