Susan and Walter are getting ready for their big summer barbecue party. Susan agrees to be the barbecue grill master if Walter buys all the ingredients. Not sure how many hamburgers to buy, Susan sends out a survey to all of the guests. When Walter checks the survey, he’s shocked. 0% of people want hamburgers?! Then he reads Susan’s survey question:
Do you want to have someone kill a defenseless animal, skin it, grab some of it, add preservatives to it, and force me to inhale its death fumes while I cry silently because it reminds me of all my animal farm friends from when I was a child?
Walter shakes his head and sighs, “Do I have to do everything myself?” He writes a new survey question and sends it out to their friends:
Do you want to support local American farmers in these troubled economic times by grilling up a traditional American juicy burger?
“There,” he thinks, “that’s better.”
So who’s right? Susan or Walter?
Actually, they’re both wrong. Both Susan and Walter have created biased questions. Biased questions influence a survey takers answers, making them change the responses they give you.
So what?
Well, when people don’t respond according to their own opinions but instead just tell you what you want to hear, everybody loses. The worst problem is that you get misleading data. Susan’s survey question will mean that there aren’t enough hamburgers at the party, and Walter’s will mean that there are too many. This will lead to cranky partygoers or wasted money at the grocery store. This may seem trivial when you’re thinking about a barbecue (though I’m sure Susan and Walter wouldn’t agree) but if you’re a business making decisions based on your survey data, the stakes can be huge. Cranky customers or product you can’t get rid of can spell D-O-O-M for your business. Moreover, biased questions will also lose you credibility with your survey takers. Biased questions are seen as unprofessional, and that’s bad for business. They may also give survey takers a negative impression of your brand or your company and this can lose you money and prestige.
How do you fix it?
Try to make people feel comfortable answering your questions truthfully. There are a few different ways you can do this:
- Remove unnecessary information. Instead of expressing your views on animal rights or American patriotism, keep it simple and direct. Something like, “How many hamburgers do you typically eat at a barbecue?” This will free people up to say what they think instead of what they think you want to hear.
- Balance the question. Present both sides within in the question. For example, “Do you want a hamburger or not?” This signals to people that they’re welcome to dissent with the way your question is phrased–no hard feelings.
- Balance the survey. Sometimes it’s hard to balance the question itself, and it has to be framed in either a positive or a negative manner. To get around this problem, just make sure that your survey has both positively and negatively framed questions in it. In Susan and Walter’s case, they could ask about both hamburger intake AND veggie burger intake.
The moral of the story: When it comes to survey writing, keep your opinions to yourself and your own biases quiet. It’s better for your data and for your image.
As for Susan and Walter’s party guests? Well, we support your barbecue item of choice, whether it be hamburgers or veggie burgers. We’re fair and balanced like that.
Questions for us? Let us know in the Comments below!
