Think of a survey as a (slightly one-sided) conversation. One person asks the questions, the other one answers them.
Great and smart surveys, much like great and smart conversations, have a natural flow and feel.
So, what can you do to ensure your survey’s respondents feel like they’re a part of the conversion?
There are two types of mistakes that people make the most frequently when creating surveys which impedes that nice and easy “flow” of conversation that’s needed:
1. Question ordering: Your questions should appear in an order that makes sense. Begin surveys with questions that are easy to answer (not overly personal) and keep related questions together. Make sure that the order isn’t introducing bias into your survey.
2. Question phrasing: Make sure your questions are phrased in a conversational manner. Clarify anything that might be vague or confusing, and try to strike a balance between sounding approachable (to make sure people understand you and feel comfortable) and sounding formal (to make sure people take the survey seriously)!
In order to make better sense of this advice, let’s drop in on Charlie and Lucy’s first date:
QUESTION ORDERING (What Not to Do):
Charlie is on a blind date with Lucy. A little nervous, a little interested, and unsure of what to expect, he walks up to her and says hello.
“Hi!” says Charlie.
“How much money do you make per year?” replies Lucy.
Charlie shifts uncomfortably and is reluctant to answer her question. He doesn’t even know her last name and she wants to know his salary?!
So how does this relate to your survey? In surveys, just like in conversations, you can’t start asking questions that are too personal or sensitive right off the bat. You need to develop a rapport with your respondents first and save the sensitive questions for the end once you’re sure that the respondent won’t drop out of the survey (for more about how to ask sensitive questions, stay tuned for our next Methodology blog post).
Quick fix: Ask sensitive questions last.
Charlie reluctantly tells Lucy how much money he makes. Then she asks him what kind of flowers he likes best, how many times he went to the gym, if he’s ever been sprayed by a skunk, and whether he believes in the American Dream.
“What does knowing how much money I make have to do with whether I’ve ever been sprayed by a skunk?” Charlie wonders.
Charlie and Lucy haven’t even known each other for five minutes and already he’s feeling a little overwhelmed and his head is spinning. He eyes the door and considers making a run for it just to avoid any more of Lucy’s crazy questions.
So how does this relate to your survey? Make sure that your beginning questions match the subject of your survey. If you tell respondents you’re going to survey them about cars, but you start off by asking about politics, they’re more likely to drop out. Moreover, throwing a bunch of unrelated questions together in one survey confuses respondents. Survey questions should progress naturally from one to another- remember that ‘easy flow’? A simple way to do this is to group related questions together. Instead of asking about cars and then social media and then planes and then TV viewing, group the transportation questions together and the media ones together.
Quick fix: Use page breaks to group related questions together. Include headings that explain to respondents why you’re asking them the questions.
“Do I believe in the American Dream?” Charlie thinks to himself.
“Is that because I told her I only make $12,000 a year and she wants to know if I think that I might make more someday so I can provide for a family? Uh- oh. I better say I believe in it or she’ll think I’m never going to make enough money to take care of her.”
So how does this relate to your survey? Past questions can bias responses to future questions! This same scenario between Charlie and Lucy could happen on a survey, where your respondent changes their answer to a question because they’re actually thinking about a previous response. This is especially likely to happen if you ask demographic type questions in the beginning of your survey. There have been numerous psychological studies showing that asking demographic questions (age, income, gender, ethnicity, etc.) at the beginning of a survey is a bad idea because they change responses on the questions that come next, and can even impair performance on tests of math and verbal skills.
Quick fix: Again, headings will help fight your respondents’ anxiety about why you’re asking. It’s also best to ask demographic questions last.
QUESTION PHRASING (What Not to Do):
Lucy isn’t finished with her list of questions for Charlie.
“Do you have a good boss at work?”
Unsure of how to answer, Charlie replies, “In what sense? In terms of his work performance? In terms of his personality? Overall?”
Lucy responds, “I mean- how well does he manage you and your team?”
So how does this relate to your survey? Here’s where surveys can be a little different from regular conversations. In conversations, we can always ask the other person to clarify what they mean. In surveys, we can’t. In surveys, unlike conversations, there are no opportunities for follow-up questions. All we have to go on is what’s written in the survey. That means you, the survey-maker, need to be as clear and as specific as possible about the questions you’re asking so that your respondents can answer them!
Quick fix: Pre-test your survey on a few people and get feedback before you send it out to everyone. If they don’t know what you mean, then you need to make your questions clearer.
Lucy’s next question is the worst one yet: “How many herbaceous graminoids do you have in your personal ecosystem?”
Charlie stares at Lucy blankly. “What?” he asks.
“Oh,” says Lucy condescendingly, “I forget that not everyone is a botanist! How much green stuff do you have outside of the place where you live?”
Insulted, Charlie snaps back, “All you had to do in the first place was ask, ‘How many trees do you have in your backyard?’”
So how does this relate to your survey? On the one hand, you need to remember that not all respondents are going to understand you if you use language or vocabulary that’s too complex. On the other hand, you also don’t want to talk down to respondents. You don’t want to make them think you assume they’re stupid. Plus, if you ask them about “green stuff” instead of “grass,” they won’t take you or your survey very seriously. So the ideal balance to strike is between being conversational while also being a bit formal – easy enough to understand but professional enough to be taken seriously. Surveys are, after all, serious business! It’s always good if your respondents enjoy taking your surveys, but you also don’t want them to feel as if they can just goof around.
Quick fix: Use formal but simple language. Again, pre-testing here is key.
Have you been having trouble structuring your survey as a conversation? Looking for some more tips like this? Let us know and post a comment below!
