Product testing: How to conduct an effective product assessment

95% of product launches fail—often because a brand pushes them to launch without stopping to find out if the product is something people actually want. Even the most experienced decision-makers struggle to make the right calls without input from their target audience.

Whether you’re looking for the safest bet or wondering if a risky concept might actually pay off, product testing can help you find a concept that meets your goals and resonates with your audience. With a product concept test, you can gauge customer sentiment and find the proven winners in a pool of different product ideas.

Product testing gives your target audience a range of options and asks them to provide feedback on each one. With product testing, you can compare metrics like purchase intent, quality, and value to find out which product concepts customers like best.

Let’s take a closer look at why you should test your product concepts, how to conduct an effective product assessment, and what you can do to get accurate feedback from your respondents.

While a great product can recover from a less than stellar launch, an impressive launch can’t save a product that nobody wants to buy. By asking for audience feedback on your product concepts before moving forward with a launch, you can identify successful product ideas right off the bat. This ensures that  you commit money and resources to the right projects.

When you use product testing, you take on a product launch with 4 major advantages:

Now that you know why it’s important to test your product concepts before settling on one, you’re ready to put together your test survey. For accurate and actionable results, follow these 4 steps:

While you can test any product concept you want, it’s best to choose stimuli that you’re already somewhat confident in based on an informal survey of coworkers and friends. To make sure your survey results are useful, choose products in a similar stage of development. If you’re testing a finished product against a product you’re still working on, the winner is obvious.

To make the testing process more manageable for you and your respondents, limit the number of stimuli in your survey. The maximum number of stimuli you should include in your test depends on whether you plan to use a monadic survey design or a sequential monadic survey design.

A monadic survey design divides up your respondent pool and presents each respondent with a survey asking for feedback on a single stimulus. When you’ve gathered enough feedback on each stimulus, you can pick a winning concept by aggregating the feedback from all your respondents.

So, if you have a sample size of 500 respondents, you ask 250 of them about one product option and 250 of them about the other.

Here’s an example of a product testing survey with a monadic design.

This survey design lets you ask more questions about each stimulus, which lets you figure out more about how  individual features resonate, and which shortcomings might prove difficult in the future. It’s also more likely to result in a relatively short questionnaire, which benefits your survey’s quality of data and completion rate. However, since you’re only showing each respondent one stimulus, you’ll need to target a larger audience. This can prove expensive and it might not be as easy to execute.

A sequential monadic survey design presents every respondent with the same survey that asks for feedback on multiple stimuli. Once you’ve collected responses from a statistically significant number of respondents, you can pick a winning concept by looking at the most common reactions to the stimuli.

In this type of survey, if you have a sample size of 500, you’d ask all 500 about both product ideas. 

Here’s what your product testing survey might look like if you use a sequential monadic design.

This type of design lets you target a smaller audience than a monadic design, which makes it more cost-effective. However, if you want to keep your survey to a manageable length, you won’t be able to ask as many questions about each stimulus—meaning fewer insights about the specifics of each product.

Learn more about the pros and cons of each design, and how you can set them up in SurveyMonkey.

Decide which metrics you want to look at

What makes a good product? The metrics you choose to measure will help you decide.

The questions you ask in your product testing survey can be as specific as you’d like. If you want to know whether your audience finds a certain feature useful, for example, you might ask about the innovativeness, relevance, and value of that feature. In general, these are some key metrics you might want to include in your survey:

The relative value of each metric you measure depends on your goals for the product. If you want a product that will sell well, purchase intent may be the most important metric for you. If your main goal is to win loyal customers, you’ll likely place more importance on quality.

We suggest using the Likert scale to test your metrics. This scale is simple and intuitive for respondents, offers consistent choices, and makes it easy for you to analyze results. If you choose this scale, your questions can follow the formula, “How (metric) is the product?” where your answer choices range from “Extremely (metric)” to “Not at all (metric).”

For example, here’s how the question might look if your metric is innovativeness:

How innovative is this product?

Sometimes, it isn’t clear from a respondent’s answers which concept they like best. Resolve this issue by including a forced-answer question that asks respondents to choose their favorite product from all the options at the end of the survey. The results of this question will also help you rank the importance of each metric to the overall likeability of your product. For example, if respondents consistently choose the product with the highest value as their favorite, you can assume value is a key value driver for your target market.

You might also want to include screener, category, and demographic questions in your survey. These questions tell you more about the people you’re collecting feedback from and let you segment your results by group. 

Your target audience can help you decide which product concepts are worth pursuing and which you should set aside. There are 2 ways to get feedback from this group of people:

Once you’ve collected your survey responses, you’re ready to compare your product concepts. You can use Top 2 Box scores to make your data clearer and your findings more pronounced. It groups together positive and negative responses to each question into a single percentage, which is easier to compare.

You also need to look at open-ended responses. Our word cloud feature can make them easier to grock.

Product testing belongs to the development and growth stages of the product life cycle, when you’re testing new product ideas and adding new inventory.

During the development stage, you’ll probably be looking for broad feedback that’ll help you make major decisions about your products. This is the time to ask respondents to choose between several very different product concepts, for example. Because everything about your brand is new at this point, it’s the best time to field broad, directional feedback on your products.

During the growth stage, you may be looking for more nuanced feedback on your product concepts. This is the time to ask respondents focused questions about new product features or niche alternatives to current products, for example. It’s also the time to think about sampling your loyal customers to see if they’re on board with your new ideas.

By following a set of best practices for product testing, you can get better feedback from your respondents. Here are 5 things to keep in mind:

That’s it! Now you understand why testing your products is important, know how to run your own test, and have some useful testing tricks up your sleeve. Now it’s time to jump into your own product testing survey.