I have a confession to make: I’ve been obsessively following the recent story about NCAA athletic conference realignment, hanging on all the details, developments, and speculation the same way a lot of people did with the Casey Anthony trial or Charlie Sheen’s public meltdown earlier this year. A lot of that is because Syracuse, my favorite college team, became a key player in the story when they joined Pitt in jumping to the Atlantic Coast Conference (ACC). But also, because it’s fun to follow along with the soap opera aspect of schools competing with each other off the field in a high stakes contest for the increased money and media exposure that comes with being in one of the elite athletic conferences. This story involves money, TV, aspiration, desperation, wheeling and dealing, winners, losers and goofy mascots. What could possibly make it more awesome?
How about a market research angle?
According to Yahoo! Sports/Rivals.com , Baylor University (which has threatened a lawsuit to prevent defections from its conference, the Big 12) commissioned a poll of residents in Big 12 states to get their opinions on the realignment issue. The poll found that, “Three-quarters (76%) of college football fans say they would be disappointed with the creation of super conferences that would eliminate historical regional conference rivalries.”
But, as Rivals.com blogger Jeff Eisenberg points out, some believe the questions on the poll were loaded — unfairly worded, either because the poll’s authors were too close to the issue to see the bias or because a deliberate attempt was being made to influence people to answer in a way that supported Baylor’s case against other schools in the Big 12 defecting to the so-called super conferences.
Here are some of the poll questions Eisenberg cites as being biased:
• In your opinion, how would college students be impacted, especially those who chose to go to schools with elite football teams, if conference affiliations required them to travel great distances to attend far away games?
• How disappointed would you be if historic college football rivalries in regional conferences were eliminated by the creation of a handful of super conferences with less regard for historic rivalries?
• I am going to read you two statements. After I read each statement, please tell me which one comes closest to representing your current opinion about college football?
- Regionally based football conferences that currently exist are the best option for student-athletes, alumni and fans of the teams competing.
- Regionally based conferences should be replaced with a small number of Super conferences that are formed irrespective of geography.
Eisenberg concludes, “The (Baylor) Bears may as well have asked whether people would prefer to see the Big 12 stay together or every felon in the Texas prison system be set free.”
Here in the Bunker, we take great care to avoid crafting survey questions that might bias respondents or influence the responses in a certain direction. It can be quite a challenge and sometimes requires a great deal of subtle wordsmithing. It isn’t uncommon for our team here to debate amongst ourselves between two different words we might use in a question, trying to decide which is the most neutral.
With that said, we’re not willing to accuse Baylor of “poorly disguised propaganda,” as the Rivals.com blog post did. Who’s to say if the question wording was intentionally biased or not? But we do think that some of the questions, as written, are leading. At the very least, they betray the point of view of those who commissioned the poll, which shouldn’t be the case. Actually, the poll question we had the most issue with wasn’t cited in Eisenberg’s blog post. It was a rating question that asked people to rate their level of agreement with this statement: “The Boards of our universities should appropriately balance the needs of their college athletics programs with their academic programs.”
The results showed that 70% of respondents “strongly” agreed with the statement and 20% “somewhat” agreed with it. Of course they did. Who is going to disagree with the concept of “appropriate balance”? It’s like asking people if they support the notion of “fairness.” The issue isn’t whether people want an “appropriate balance” – everyone does. The real issue is how they define what’s appropriate and what’s balanced. This poll doesn’t seem well-designed to discern those answers in the way that an objective study of the issue probably ought to.
We in the Bunker, believe this story raises a very important, fundamental issue in market research. In order to obtain good answers, you have to start with the right questions. Biased polls might tell you what you want to hear (or what you want to be heard), but they don’t shed any light on the issue and they leave you open to accusations that your research is nothing more than self-serving propaganda.
And can I even tell you how much I’m looking forward to seeing Syracuse play Duke and North Carolina in basketball every year?
[…] approval rating, or the results of some other news poll about a current issue. And of course sports and entertainment websites are littered with unscientific reader polls, that usually consist of a […]