In the 2016 book, Market Research in Practice – An introduction to gaining greater market insight (3rd Edition), our B2B International experts commented on the importance of keeping questionnaires engaging, to keep both response rates and response quality high.
Three years down the line, in a world that is busier and engagement is harder, this article collates some top tips for keeping respondents engaged, including some innovative questioning styles trialed and tested in recent months.
1. Keep It Easy
Questionnaires can be conducted through different methods, including online and telephone. Whilst each method has its own advantages and disadvantages, it is important to consider how your question will be asked to understand how easy it will be for a respondent to answer.
For example, asking respondents to select the 3 most important attributes from a list of 15 is a relatively simple task when all this information is presented on screen, but quite a challenge during a telephone interview.
Answering this question in a telephone interview requires two stages: the respondent needs to remember the attributes before they can decide what is most important to them, both of which require working memory1.
Our working memory is a temporary storage system of information, used in language comprehension, reasoning and problem solving2. Including a long list of attributes makes the storage and recall more difficult and whilst the capacity of working memory is still debated, research suggests we are able to hold somewhere between 3 -7 items2.
Combining these insights with our market research experience, we recommend asking respondents to rank no more than 8 attributes. This means the list is long enough to assess a variety of attributes, but not so long that it is too demanding of respondents.
2. Keep It Varied
Consistency is important in any questionnaire – using the same scales across different questions means respondents become familiar with the formats, meaning more time is spent answering the question than trying to understand the scales.
However, this familiarity can sometimes lead to ‘flat lining’ – where respondents give the same score across attributes, primarily because they are no longer engaged in the survey.
Including a mixture of closed, scoring questions with open questions keeps the questionnaire varied and helps to prevent disengagement from respondents. Open questions can help bring topics to light that may not have been considered and are particularly useful when looking at upcoming challenges or trends.
3. Keep It Innovative
At B2B International, we strive for innovative and creative question formats to keep respondents engaged.
One format we have used frequently in recent months is the use of emoji’s as rating scales. For example, a respondent would be presented with a range of emojis and asked to select which one best represents their experience, or how they feel about a statement.
This design is much more visual when compared to the more familiar and traditional 1-5 scales, and research indicates that respondents find the use of emojis fun and more engaging3.
In addition, the simplicity of using emoji’s made the decision making easier, which in turn made it quicker to complete3. Emoji’s are widely used and their familiarity makes understanding easier for respondents who may be completing the questionnaire in their second language3.
Whilst there may be drawbacks to this style of question, it does present a more fun, engaging alternative that can be universally understood across languages.
1. Baddeley AD, Hitch G. Working memory. In: Bower GH, editor. The psychology of learning and motivation. Vol. 8. New York: Academic Press; 1974. pp. 47–89.
2. Cowan N. (2010). The Magical Mystery Four: How is Working Memory Capacity Limited, and Why?. Current directions in psychological science, 19(1), 51–57. doi:10.1177/0963721409359277
3. Alismail, S., & Zhang, H. (2018, January). The use of Emoji in electronic user experience questionnaire: an exploratory case study. In Proceedings of the 51st Hawaii International Conference on System Sciences.