The wrapping paper strewn all over your living room floor and the leftovers in the kitchen make it pretty clear that the holidays are over! As you clear away the wrapping paper, you think back to the holiday party you hosted last week. Some people seemed to really enjoy themselves, but others looked distinctly uncomfortable.
You decide to create a survey to figure out whether your guests enjoyed themselves or not. You form the first question that comes to mind:
Was my holiday party fun this year?
☐ Yes
☐ No
Why is this bad? There isn’t enough variance. Variance measures how spread out your responses are, or how big of a range there is in your responses. There are only two categories of responses here: Yes and No.
So what? Well, variance gives you more information about how people think, or feel, or behave. The more variance you have, the better a picture you can paint about what was going on inside someone’s head when they answered the question. There are two main ways that a yes/no question fails to do this:
- First, it fails to capture the people who are on the fence. These are the people who don’t like extremes. The ones who get “twist” ice cream because they can’t decide between chocolate and vanilla. They didn’t love your party but they didn’t hate it. Give these people only two options and they’re likely to quit out of your survey without completing it. This will bias your sample—and your results!
- Next it fails to capture the true nuances of people’s opinions. Opinions are rarely completely black and white—there are shades of gray. Your mom might have liked the party a little, but not much. Your roommate, on the other hand, might have thought it was the best party she’d ever been to. But both your mom’s lukewarm feelings and your roommate’s extremely positive feelings would be lumped together in the “yes” category. These results would likely be misleading if you use this feedback to plan your mom’s 50th birthday party next month.
Does this fix it? A lot of you regular blog readers out there already know that Yes/No questions are to be avoided and that you should create a Likert scale type question.
So you go ahead and create this:
How much fun was my holiday party this year?
☐ Extremely fun
☐ Very fun
☐ Moderately fun
☐ Slightly fun
☐ Not at all fun
Problem solved! Or is it?
But then you make one small mistake. When you sit down to do your data analysis, you decide to combine the top two responses and the bottom three responses to give you a simple indication of how many people liked your party and how many people didn’t.
So now your question looks like this:
How much fun was my holiday party this year?
☐ Extremely fun / Very fun
☐ Moderately fun / Slightly fun / Not at all fun
This avoids the first flaw of a Yes/No question in that you have defined what “Yes” means after the fact, but it fails the second part. You’ve lost all the variance you worked so hard to create!
Again, so what? Still not convinced that you should care? Read on, survey fans!
- Keeping your variance lets you look at the distribution of responses. You’ll be able to see how people’s opinions are spread throughout that range. You might find, for example, that everyone had the same opinion, or that most people loved it and very few people didn’t, or that everyone’s opinions were spread out very evenly.
- It lets you do more detailed analyses. For example, let’s say that the problem with your party was that it appealed too much to younger people. Having both your mom and roommate’s responses together as “yes” completely obscures the main reason for your party’s success (or failure) in people’s eyes.
Want a better simple fix? We get it, sometimes you just want a really simple statistic to present. For that, we suggest computing the mean of your responses. Mean is really just a fancy word for average, and this can be done by assigning a number to each point of your scale:
☐ Extremely fun 5
☐ Very fun 4
☐ Moderately fun 3
☐ Slightly fun 2
☐ Not at all fun 1
Then all you do is take the average of all of the responses you get and, voila! An average response to your party is right there in front of you.
One little heads-up on “means”… When calculating an average using response options that are originally words, it assumes that the distance between each response option is the same. For example, the space between “Extremely fun” and “Very fun” is assumed to be the same as the distance between “Moderately fun” and “Slightly fun.”
The great thing about using a standard Likert scale as we recommend, we’ve already done all the tough research for you to make sure that this assumption is indeed true! If you decide not to? Then we say interpret with caution, survey friends!
Looking for even more info on snazzy Likert scale type questions? Let us know in the Comments section below and don’t forget to check out even more on Likert scales at our Help Center!
