4 leading types of bias in research and how to prevent them from impacting your survey

Bias is the mortal enemy of all surveys, and as a survey creator it’s important to guard against it to make sure you get reliable results. Over the years, we’ve offered best practices for designing surveys that address different types of bias in research, such as unbiased wording, structure, and styling. But if you’re not careful, there are a few ways you can still introduce bias without even knowing it.

Of all the different types of bias in research, many come directly from the survey writer. This bias is sneaky. It’s caused by survey creators who innocently influence the results in an effort to reach their desired outcome. But in doing so, they influence the credibility and value of the results themselves.

Can you avoid this kind of bias?

Yes! Here are the top 4 types of bias in research and tips for designing your survey in ways that proactively address them:

It’s impossible to get the right answers if you ask the wrong questions. Unfortunately, survey results are easily compromised by questions that fall short of capturing the entire scope of a survey’s issue. Say, for example, your survey was created to understand your employees’ favorite type of pizza. You ask, “Do you like pepperoni, meat lovers, or vegetarian pizza the best?” Though there are many other types of pizza, they did not come to your mind and were left out of the question. Now instead of measuring the most popular pizza, the study measures the preference between these three types.

Tip: Exploratory research is the best way to make sure your questions are exhaustive and on point. By first surveying a small group with open-ended questions on your subject, you’ll gain a better perspective of the scope of your survey topic, and be less likely to overlook options that may matter to respondents. You may also want to review similar surveys to learn what categories and topics were popular with respondents in the past.

Choosing your respondent group may seem like a no-brainer, but it often leads to something called selection bias. When conducting a survey, it’s imperative to target a population that fits your survey goals. If you incorrectly exclude or include participants, you may get skewed data results.

Usually this bias happens when you lack of a clearly defined target population. For example, say you want to limit your survey to people with low economic standings. This population could be defined in many ways: people with low income, people who lack disposable income, or people who have a low net worth after taking into account their property, income, and debt. Each of these three descriptions can successfully be used to address the broad population you hope to reach. But, each definition could provide different results for your study.

Tip: To avoid surveying the wrong people, make sure you clearly define the respondent requirements you need to meet your survey objectives before beginning your project. This step will give your survey results a proper scope. Also remember to be specific in your reports and findings when referring to your population. Using broad terms like poor, rich, large, or small can lead to misinterpretation.

Some surveying methods can make it difficult, or even impossible, for certain people to take part in your study. For example, if you survey commuters you meet walking around on the street, you might not get a representative sample of people who drive or ride bicycles. By excluding potential respondents in a non-random way, you can instill bias into your survey if the people who aren’t part of the panel have views that differ than those who are.

Tip: The best way to limit this type of researcher bias is to give all potential respondents an even chance to participate in your survey. In our commuter example, you might be better off sending an online survey to everyone who lives in your town, or asking some local businesses to send your survey out to all their employees.

This form of bias is introduced when raw data is transformed into misinterpreted findings. Usually it’s a case of inappropriate or inaccurate statistical techniques, which lead to the incorrect interpretation of the survey results. For example, bias can come into play when a survey creator gets excited about a finding that meets their hypothesis but overlooks the fact that the survey result is only based on a handful of respondents.

Make sure that your results have the sample size you need to make conclusive decisions by using our sample size calculator.

Tip: Most often, this form of bias is caused by gathering information and then later developing your data analysis strategy. To avoid this type of bias, create a data analysis plan before you write your survey. Then write questions that you know will work well with the analysis you have in mind. For example, use a multiple choice question if you want to quantify your results. Finally, take note of the different analytical tools available in your survey software beforehand. That way, you’ll know the types of analysis that are—and aren’t—possible before you create your survey.

Avoiding these four types of bias in research may seem difficult at first. But if you can remain true to your survey’s purpose and develop a firm understanding of the topics covered in your survey, you’ll be well on your way towards eliminating each of these types of bias in research.

Put simply: You need to do some planning before you start your survey. Take a second to think about each of the four points above and whether your survey plan addresses them. Once you’re sure that your research methodology is sound, you can rest easy that your final results won’t lead you astray.