How to use best-worst scaling to make better decisions

Best-worst scaling, also known as MaxDiff (maximum difference) analysis, is useful when gauging preferences about messaging, brand names, product features and more. It’s easy to use and offers clear insights into what customers truly like and dislike. Let’s take a look at best-worst scaling and how you can use it to make better decisions for your business.

Best-worst scaling is a type of survey research conducted to understand the relative importance of attributes such as product features, packaging, messaging, etc. in the eyes of your target market. By identifying what consumers really want most, you can focus your business’s investment of time and capital on efforts that will appeal to your target market.

Best-worst scaling asks respondents to choose from several options at once, selecting only the best and worst options. This type of survey question allows you to collect the information you are seeking quickly and definitively. There’s no guesswork about what respondents mean when they choose a score near the middle of the range as in Likert scales or rating scale questions. Your respondents simply choose the most and least important options to them. MaxDiff questions can be asked in a single survey or as part of a longer questionnaire.

Two terms are frequently used in best-worst scaling:

How to conduct your best-worst scaling study: 

Best-worst scaling is very similar to conjoint analysis for determining respondents’ preferences for various items or features. However, MaxDiff analysis is easier to use and less refined than conjoint analysis. 

Here are some examples of the two methods for comparison:

Conjoint analysis studies mimic shopping trips, where participants review products, features, attributes and prices to make purchase decisions. The analysis is complex and considers multiple factors.

For example, you could ask “If you were in the market for a new smartphone, which of the following would be most appealing to you?”. Respondents would compare brand, price, storage and more. 

In best-worst scale survey questions, respondents are asked to choose the least and most important factors within the answer options. Data analysis is faster and cleaner.

For example, you could ask, “If you were in the market for a new smartphone, please indicate the feature that would be the most important in your purchase decision and which feature would be the least important.” You would then list various features, such as camera, display, face ID, etc., and respondents would decide which feature was most and least important. 

As you can see from the examples, both methods aim to discover what customers prioritise when purchasing a smartphone. The best-worst model data will reveal the most preferred feature and the least preferred feature. Conjoint analysis shows how much each feature influences the final purchasing decision. 

These methods are often used together to create a more detailed picture of what customers want and how much they are willing to pay for it.

In addition to the ease of creation and data analysis, best-worst scaling has several advantages:

Best-worst scaling questions are easy for respondents to answer. They are, in effect, simulating real-world behaviours in making choices and trade-offs, eliminating options that they don’t feel strongly about. Answering several of these questions makes the strength and importance of each choice known.

By forgoing a ranking scale, best-worst scaling avoids biases from cultural differences – as some cultures have associations with different numbers – or perceptions of ratings. Respondents may also have list order bias, where they indicate that all features are equally important. When bias occurs, the business may waste resources on improvements or features that aren’t truly important to customers.

MaxDiff survey data is well suited to creating statistical models. These models make it possible to quantify preferences and understand what your target market values.

Determining which features are deemed best and worst by your customers will provide you with actionable insights for product features and improvements, new products and obsolete product features. Use this data to give your customers what they truly want and value. This will result in increased customer satisfaction overall.

As with any process, there are some downsides associated with best-worst scaling.

Depending on the number of attributes you are testing, it may take respondents more time to take your survey. Shorter surveys tend to have higher response rates. 

MaxDiff analyses measure respondents’ preferences based on the attributes presented in relation to each other. This does not account for whether the attributes offered in the survey are good or bad based on an absolute perspective.

Best-worst scaling is typically used to optimise product features rather than the product as a whole. For example, your MaxDiff survey may show a definite preference for one feature, but in reality, customers may not be willing to pay a higher price for the feature. To dig deeper into your results, perform a conjoint analysis to test the product as a whole.

Is the level of preference indicated for an attribute conditional upon the alternatives that it is compared to? This is the context effect. For example, if you’re conducting a best-worst scaling smartphone study and your attributes are Samsung, Nokia, Google, Sony and Apple, respondents are probably thinking about Google in terms of hardware. But if you’re doing a study that includes Google and Yahoo in its attributes, it’s more likely that people will think of Google in terms of its search engine.

It’s important to be aware of the context effect even though it is impractical to try to avoid attributes because of this possibility.

The use of standard rating questions can lead to user scale bias, scale meaning bias and lack of discrimination between answer choices. MaxDiff surveys eliminate the issue of bias because respondents are asked to choose their preference rather than to rank or rate.

As we mentioned in our earlier description, best-worst scaling is closely related to conjoint analysis. Conjoint analysis is used to determine how different features would affect your respondents’ purchase decision, whereas MaxDiff asks respondents which features are most important to them. To get a clear picture of your customers’ overall preference, use best-worst and conjoint analysis together to combine and compare the results.

Best-worst scaling can be used for many purposes. Primarily, it is used to identify preferences regarding a list of attributes in a set. It’s a fast and easy way to obtain important information, especially if you have a small marketing budget. Best-worst scaling is a very effective method for determining what your target market really wants.

Best-worst scaling surveys are valuable tools for the product development of upgrades. If your product sales are flagging and you’ve decided you need to update the product, a MaxDiff survey can help. Rather than deciding internally which features you think your customers want, present your features (attributes) in sets and find out what is truly important to them.

Is the name that you’re considering for your product appealing to your target market? Perform best-worst scaling and analysis to obtain information about which names are preferred by customers.

To take name testing further, use best-worst scaling to find out how appropriate the names that you’re considering for your new product are in light of your target market’s perception of your brand. 

MaxDiff surveys can be used for multiple purposes, including:

Now that we’ve covered why you should use best-worst scaling, let’s look at some examples.

Example 1

Question 1:

Think about what would make you choose one restaurant rather than another one. Considering these features, which is most important and which is least important? 

Question 2:

Think about what would make you choose one restaurant rather than another one. Considering these features, which is most important and which is least important?

Example 2

Question 1:

When choosing a hotel, what are the most and least important factors in your decision?

Question 2:

When choosing a hotel, what are the most and least important factors in your decision?

Example 3

Question 1:

Below are names for our new line of cookware for children. Please indicate which name you think is the best fit for our brand and which you think is the worst.

Question 2:

Below are names for our new line of cookware for children. Please indicate which name you think is the best fit for our brand and which you think is the worst.

Helpful tips:

Once you’ve gathered your best-worst scaling data, you need to analyse it before putting it to use.

There are four main types of analysis for MaxDiff:

No. of times the attribute was selected as best - No. of times the attribute was selected as worst
No. of times the item appeared

Data may be presented to your team, management or other stakeholders in bar graphs, pie charts or other visualisations that make the information easily digestible. 

With the data from your best-worst scaling to hand, you can make decisions about everything from prioritising improvements to choosing a product name. Because your decisions are based on data from your target market, you can avoid making choices that will flop. This will ultimately yield better responses to your decisions. 

For example, if your study revealed that an air fryer was the feature that customers most wanted you to add to your current toaster oven and you acted upon that data, you would see an increase in sales of your new model with the air fryer.

Use best-worst scaling to determine your target market’s preferences in terms of names, packaging, product features, improvements and more. And make informed decisions that your customers will love.

Learn how you can start gathering insights in as little as an hour.