Matrix questions – or, in practice, multiple questions presented on a grid – remain a staple of survey design and were used in one out of four surveys in 2024.
For survey creators, these bundled questions are easy to write and program. For respondents, they’re generally easy to interpret (and answer) since the scales and answer options stay the same across all of the items.
However, the apparent simplicity of this question type masks some significant downsides. In order to collect high-quality data, you’ll need to be aware of some of the main pitfalls of this question type, especially since taking surveys on mobile devices continues to be on the rise.
Question writers are often guilty of overusing matrix questions due to the fact that they are so easy to create. They offer too many response options and too many items or rows. This makes it difficult for people to read and can encourage poor survey-taking behaviour, such as straight-lining (where respondents select the same response for each item without careful consideration of each row) or randomly selecting responses just to escape the matrix!
Tips for making sure you don’t get caught in the matrix
Are you ready to take a look at some example matrix questions? We thought so! Let’s suppose that you manage a high-street clothes shop and you’ve written a customer satisfaction survey, which you send out to everybody who’s purchased something from you in the past week. Here’s your first question:
At first glance this question doesn’t seem too bad, but there are a few things that you can do to immediately improve this matrix question. You should start by removing the unnecessary ‘N/A’ and also the numbers to make the response options shorter.
If you would like to use numbers in your analysis, try the ‘use weights‘ option when you are creating your survey. This will unclutter your survey, and you’ll still be able to find the average rating in Analyze:
4 quick tips for writing good matrix questions
- Keep response options short. (Response options are the columns of possible responses.)
- Try to try to limit the number of response options displayed to five or fewer.
- Keep the wording of the items short. (The ‘items’ are the rows.)
- Again, try to limit the number of items displayed to five or fewer.
Don’t worry if your matrix question doesn’t fully follow the guidelines above; you can still use the questions that you’ve written. In fact, the example that we have given doesn’t exactly abide by those rules either. You can improve the experience for your respondents and get better data by doing one of two things: either splitting your matrix into individual questions; or splitting your large matrix into several smaller matrices.
Here’s a look at the individual question format:
And here’s a look at smaller matrices:
Another thing to bear in mind is mobile devices. We’ve discussed them a lot recently because we’ve noticed that more and more people are responding to surveys using a smartphone or tablet. The horizontal formatting of matrix questions isn’t ideal for smartphones. Take a look at the matrix question from our customer satisfaction survey above when it’s displayed on a mobile device:
What do you think to it? Well, you’ve probably guessed that we aren’t impressed! If you think that most of your responses are going to come from mobile respondents, regardless of the size of the matrix, you may want to consider not using that question type and splitting this into individual questions instead. At the very least, you should do some testing to see whether the matrix question displays properly on your mobile device and the buttons are easy for people to click or tap. Here’s our suggestion:
There you have it, matrix fans. Don’t worry about getting stuck in the matrix, because we have you covered.
