How to use UX surveys to test ideas

Do you have a solid understanding of how users interact with your product? What about how they feel while using it, or whether it’s everything they hoped for when they passed that pivotal point of purchase? A lot transpires after that glittering moment when a potential buyer makes a decisive move, opts for your product and becomes a new user. The real test comes when the customer begins to live, work or play with the product, developing a firsthand view of whether it’s delivering on its initial promise. 

The thoughts, feelings and actions that transpire throughout this process are an important part of what is known as the user experience (UX). Capturing those sentiments quickly and accurately is vital to the success of your organisation. 

You need to know if the experience that your customers have with a product or service is fraught with problems or simply doesn’t meet certain needs or expectations. Equally important is learning that the user experience has been seamless, meaningful, intuitive and satisfying. 

For your current and future business, user experience matters – a lot. And the best way to assess user experience, and test out new ideas for enhancements and improvements, is by consistently using UX surveys to collect valuable user feedback.  

Effective UX surveys not only identify issues and problems, but also provide insight into what your users really want and need. They can provide data and direction for developing new and innovative products and generate feedback that confirms whether upgrades and changes are hitting the mark with users.   

Adhering to some proven best practices can help ensure that you get the most from your UX surveys. The most successful UX surveys are simple UX surveys; when they’re short and to the point, it encourages greater participation and makes analysis easier. If possible, it can also be beneficial to embed on-site surveys (surveys that pop up or appear directly within a webpage) to make them more relevant, quick and easy to complete. 

As you plan and create UX surveys to test ideas, keep these additional tips in mind:

When conducting a UX survey, it’s important to consider the unique benefits of employing both open-ended and closed-ended questions to make sure you capture the full range of useful data and insights into how your users are experiencing your product or services. 

Typically, closed-ended questions elicit more straightforward responses. As a result, it’s easier to comb through the data and understand what works for your customers, and what doesn’t. 

Meanwhile, open-ended questions that allow respondents to expand upon their answers offer greater context and can generate thought-provoking ideas on how to improve a product or address a problem identified by many of your customers. 

Examples of closed-ended UX questions include:

These questions require respondents to choose from pre-set answer options. They generate quantitative data, making them perfect for companies looking to collect conclusive, statistically significant results about their user experience.

Meanwhile, qualitative data is collected by asking open-ended questions that dig deeper into the user’s behaviour, actions and motivations. You gain insights into how users are thinking and reacting when you ask open-ended questions such as:

These questions are accompanied by a textbox, so the user can explain their opinions, feelings and experiences in their own words. Using text analysis features such as sentiment analysis and word clouds can help save analysis time and generate valuable insights based on qualitative data. 

If you’re going to ask people about their experience with your product, service or website, you need to make sure they’ve interacted with your offering or have enough experience to answer your questions. To help ensure that the right questions are being asked of the right users, it can be useful to apply logic to your questions, which allows users to avoid answering questions that are not relevant to them. 

Also referred to as question skip logic, this method is applied to closed-ended questions. It allows you to skip respondents to another question or a different page based on their responses to a particular question. Depending on their responses, respondents are either skipped to a new page, the survey concludes or the respondent is disqualified

Ensuring that survey respondents are only seeing questions that are applicable to their experience makes it easier for you to analyse the results once the data has been captured. 

Bias can skew your survey results, either by influencing respondents to favour your most desired answers or by impacting your analysis. This can set off a dangerous domino effect in which bias generates bad data; the bad data taints your analysis; and your flawed analysis leads to costly business decisions.

With that in mind, you should be constantly vigilant about preventing any sort of bias from creeping into your UX survey questions or analysis. The challenge with stamping out bias is that you may sometimes not be aware that there’s bias in your questions or how you view your results. Gaining greater awareness about the different types of biases can limit the chances of your survey results being skewed. It’s good practice to get a diverse group of people to review your questions to see if they detect any bias.

Here are some of the cognitive biases that you should avoid in your surveys:

Confirmation bias: The most common bias, confirmation bias generally occurs during survey analysis. It’s when a researcher looks for patterns in their survey data to justify preconceived beliefs, while discounting or disregarding information that challenges those beliefs. The best way to avoid this is by creating survey questions that allow for easy interpretation of results, having a clear plan for survey analysis or multiple analysts, and checking and rechecking positive and affirmative responses. Dig deeper and ask questions to find out more about user behaviour, actions and decisions. Challenge yourself consistently throughout the research process as well.

Unbalanced scales: Unbalanced scales can also influence the way users respond to UX surveys by limiting the choices available to them. For example, don’t ask “How much did you enjoy your experience with this product, on a scale of 1 (enjoyed it a little) to 5 (enjoyed it a lot)?” In this case, the question is okay, but the scale is not. The wording of the question clearly infers to the user that they did, in fact, enjoy their experience to some degree. Yet the truth is they may not have enjoyed it at all. They might have absolutely hated it. If “enjoyed it a little”' is the closest answer option they have to express their dislike of the experience, you won’t get a true understanding of the user experience. 

A fix for this UX question would be to leave “enjoyed a lot” as the highest end of the ranking scale and to make “didn’t enjoy at all” the answer option at the low end. When you ask users to rate their experience, make sure you give them an inclusive and logical scale of answer options so that the results are as accurate as possible. If you don't, you run the risk of presenting them with leading questions that won’t lead to the results you need. 

Framing effect: How you present – or frame – your questions will ultimately set the tone of your UX survey. For example, asking “What did you like/dislike while using the product?” can potentially make respondents think only of the clear-cut positives and negatives of their experience, which often only represents a fraction of the whole story. Although their positive and negative feedback offers valuable big-picture insight, it doesn’t provide a full landscape of what other considerations and options factored into the consumer’s interaction with the product.

An unbiased alternative is to ask more neutral questions, such as “Can you describe how you feel when you use this product?” This style of open-ended questioning gives the respondent enough room to think about the entire experience, from start to finish, and doesn’t limit them to thinking only about the most enjoyable or problematic aspects of a product.   

Screeners are basically the bouncers of the survey world. Asking screener questions at the start of your UX surveys will help you home in on your target audience and eliminate those who do not fit the persona you are trying to reach. The screening process saves you – and your audience – time by asking a set of potentially disqualifying questions before the actual survey begins. The way respondents answer lets you know whether they have the relevant experience to proceed through your UX survey.

There are two types of screening questions that you can ask. Behavioural screeners are questions based on behaviour. For example, let’s say you’re looking for website feedback and want to survey the most frequent visitors to your site. A screening question such as “How often do you visit this website?” would reveal who visits every day, several times per week, a few times a month, rarely or never.

In this instance, screening out those who only visit the site a few times a month, as well as those who rarely or never visit, can help you develop a survey group that has significant experience with your website and can give you the most relevant and useful data. 

Meanwhile, industry-specific screeners weed out respondents who may have a bias towards your product or service because they work in the same or similar industries as you. In a sense, they know too much, or may have relationships or preconceived notions that would influence their responses. A common example of an industry screening question would be “Do you or anyone in your family work in the following industries?” By disqualifying anyone who selects your industry in the answer options, you can eliminate or minimise potential bias.

Learn more how to qualify or disqualify respondents with SurveyMonkey’s Screening Questions Guide.

There are three distinct phases of UX surveys, each of which serves a valuable role in capturing user feedback for assessing, developing and improving products and services. Regardless of what sort of idea you want to test, these phases can help you make customer-centric decisions.

The Think Phase is where innovation can happen. User feedback captured during this phase helps you brainstorm new opportunities and reveals qualitative insights about your product or service. An example of an effective Think Phase question is: “What is one thing you wish this product could do that it isn’t capable of doing already?”

This question helps you gain insights and ideas from users who are familiar with the product’s benefits and shortcomings. They may come up with ideas that would benefit all users and are worth considering for future iterations.

The Make Phase helps you to prioritise UX changes in your product roadmap so you deliver the most impactful improvements as quickly as possible. An example of a useful Make Phase question is: “On a scale of 1-10, how would your use of our product be affected by [feature enhancement]?”

With the results from this type of question, you can make the case for certain product enhancements to move to the front of the queue. This clarifies where your focus should be, based directly on data from users who know your product well.  

The Check Phase measures if recent changes to a product or service have improved or worsened user experience. An example of a good Check Phase question is: “How has [new feature] affected the frequency with which you use our product?”

This type of question draws feedback that directly relates to a recent feature enhancement and can help confirm whether the change is achieving the desired effect. Conversely, if you get negative feedback, it provides you with the opportunity to expedite any additional upgrades or tweaks to address customer concerns.

When it comes to user experience, you want to make sure that you are meeting your customers’ expectations and developing top-notch products or services. Deploying UX surveys can improve how you deliver for your customers, which, in turn, can improve your customer relationships, your brand image and more. SurveyMonkey provides helpful resources to help you create UX surveys so you can gain more insight into user behaviour. Create an account so you can have access to every resource and tool that SurveyMonkey offers and choose a plan that works best for you and your business.