We talk a lot about mobile optimization for surveys, but we don’t always get into the nitty-gritty of why it’s so important.
Sure, more than 35% of SurveyMonkey respondents are now completing surveys on mobile devices. But how do we know that their experience is any different on a smartphone than on, say, a desktop computer?
Think about all the good and bad experiences you’ve had using your phone’s apps and web browser.
Mobile-optimized actions—like using your map app to find that hot new restaurant—are easy. It probably took just a few taps to get the information you needed.
But when you try to find the dinner menu? You’ve got to use your browser to find the restaurant website, click through tiny text links using your big ol’ finger, and go through several pages before you even find it. Then it’s a matter of pinching in to expand the menu, swiping left, then right, up then down, to view it bit by bit by…
Does that make you want to throw your phone across the room, or at least never visit that website again? That kind of frustration is exactly what we want SurveyMonkey customers to avoid.
Let’s get to the research
For completed surveys, we can measure that frustration in minutes and seconds.
Web surveys collect paradata—information about how the survey was completed—that can provide a lot of insight into the survey experience itself. For example, paradata can tell us what device a survey was taken on and when a survey was begun and completed.
Armed with that data, we know the time it takes to complete a survey on a mobile device compared to how long it takes to complete the same survey on a desktop or laptop computer.
But before we dive into that data, let’s take a look at what the academic literature on surveys says!
Web surveys take longer on mobile devices. Pretty much every published article on this topic agrees, from those in the very early days of web surveys to those examining modern, mobile-optimized surveys. It doesn’t matter if respondents were randomly assigned to use a particular device or allowed them to use whichever they prefer—mobile just takes longer.
Sometimes they can take a lot longer. Most studies find that surveys completed on mobile devices take about 50% longer than those on PCs. Some found that this could be as much as 2 or 3 times longer.
Scrolling is the culprit. Couper and Peterson (2016) propose several possible reasons for why surveys take longer on mobile, including:
- Slower connection to the server on cellular networks compared to high-speed internet
- A worse user experience reading and responding to questions on a small screen
- More distractions for users who are responding on-the-go.
Importantly, they find evidence that the biggest time drag for mobile surveys comes from something that survey designers actually have some control over: the survey-taker experience.
Matrix questions are particularly problematic. Their dimensions often force respondents to scroll both up and down and left and right, and the smaller screens of cell phones make this problem even worse.
SurveyMonkey findings and recommendations
Does SurveyMonkey data back that up? Yep.
Our Research team ran a survey where we asked more than 3,000 SurveyMonkey Audience respondents in the U.S. 21 very straightforward questions (including one 5-question, 5-response option matrix question) 1 open-ended question, and 1 question that included images.
We found that respondents who completed the survey on a mobile device took 11% longer than respondents who completed the survey on a computer (6.28 minutes to 7.01 minutes).
So, what does this mean for you the next time you’re designing a survey?
- Consider alternatives to matrix questions. They cause the most delays with scrolling and resizing.
- Including images or videos can make your survey take extra time to load for respondents who are answering on their phones on-the-go.
- Test your survey on different devices (PC, Android phone, iPhone) to see how different questions show up.
- Keep your survey short, as mobile respondents are more likely to drop out as it takes them more time to complete.
Check out our other resources for mobile surveys, such as:
Citations:
Couper, M., & Peterson, G. (2016). Why Do Web Surveys Take Longer on Smartphones? Social Science Computer Review, 1-21.
Questions, comments for Laura? Let her know below!
