Products

SurveyMonkey is built to handle every use case and need. Explore our product to learn how SurveyMonkey can work for you.

Get data-driven insights from a global leader in online surveys.

Explore core features and advanced tools in one powerful platform.

Build and customise online forms to collect info and payments.

Integrate with 100+ apps and plug-ins to get more done.

Purpose-built solutions for all of your market research needs.

Create better surveys and spot insights quickly with built-in AI.

Templates

Measure customer satisfaction and loyalty for your business.

Learn what makes customers happy and turn them into advocates.

Get actionable insights to improve the user experience.

Collect contact information from prospects, invitees, and more.

Easily collect and track RSVPs for your next event.

Find out what attendees want so that you can improve your next event.

Uncover insights to boost engagement and drive better results.

Get feedback from your attendees so you can run better meetings.

Use peer feedback to help improve employee performance.

Create better courses and improve teaching methods.

Learn how students rate the course material and its presentation.

Find out what your customers think about your new product ideas.

Resources

Best practices for using surveys and survey data

Our blog about surveys, tips for business, and more.

Tutorials and how to guides for using SurveyMonkey.

How top brands drive growth with SurveyMonkey.

Contact SalesLog in
Contact SalesLog in
Survey Platform

Statistical significance: see how close you are to the truth

Statistical significance: see how close you are to the truth

Last month, we gave you Basic Statistics so you can see the mean, median, mode, minimum, and maximum alongside your survey results in an instant.

Today, we want to show you SurveyMonkey's newest feature, Statistical Significance, where we do the advanced math for you to help you understand whether or not the results you’re getting are statistically significant.

What is statistical significance? A statistical significance tells you whether one group’s answers are substantially different from another group by using a statistical formula. If something is statistically significant, that means the numbers are reliably different, which is very helpful when you want to see if two percentages are actually different given the sample size (check out our sample size calculator) of each group.

In other words, based on the level of statistical significance you see, it means your results are either closer or further from the truth. (“Significance” here doesn’t mean “important.” It means “probably true” and not due to chance.)

The Statistical Significance feature works in combination with our Compare feature. When you apply a Compare rule, if you show Statistical Significance, we will display new information right in your data table.

You can turn on the setting to show Statistical Significance when you create your Compare rule. Just select your answer options to compare, then set the Show Statistical Significance switch to On:

You can also switch the feature on and off by going to Customize on your chart, clicking the Display Options tab, and using the Statistical Significance checkbox:

When you look at your results (in the Summary tab), you will notice some new information in the data tables. For example, here’s a data table where we’ve compared the answers to “Do you plan to use our product more or less in the next 12 months?” with the answers to “How long have you used the product?”

You can see right away that we’ve highlighted several of the cells in the table. These are the cells that have statistically significant differences. The letters in the cells tell you from which cells they are significantly different. (We added letters to the group labels in the left column, so you can tell which letter applies to which group.)

So, in this example, you can see that your new users (less than 6 months) and your oldest users (over 2 years) answered “More than before” at a significantly different rate.

In the case of the answer option “Less than before,” you can see there is a statistical significance between the new user group and both the 1-to-2 years and the over 2 years groups.

If you roll over the cells, you will get more detail on the statistical significance:

But what does this tell me?

The shading will help you come to the conclusion that more veteran users are more likely than newbie users to increase their usage of your product. Even though the percentages are different from one another—because the boxes for the two middle groups (6 months to 1 year, 1 to 2 years) aren’t shaded (indicating that they aren’t statistically different from any group)—you can be fairly confident that they aren’t any less likely than the seasoned vets to increase their usage of your product.

While Statistical Significance can tell you what is statistically different, it does not necessarily mean that the difference is meaningful. It’s up to you as the interpreter of the results in Analyze to determine if that statistical significance is important.

Running an AB test? Calculate the statistical significance of your results in seconds with our AB Testing Calculator!

Now you have all the tools to find out exactly what you need to know about your audience.

If you ask the right questions, get enough responses (SurveyMonkey Audience can help you with that) and then create Compare rules and show Statistical Significance, you can find out:

Note: The Statistical Significance feature is available with certain paid plans. Have questions for our survey pros? Let us know in the Comments!