I recently discussed the importance of getting Voice of the Customer (VOC) feedback and common methods, such as surveys, used to understand customer perceptions and expectations across touch points. To be effective and acquire actionable insights, survey questions must be designed following best practices. I also recommend a “test & learn” approach.
Writing and launching surveys is a science and an art. I have seen cases of surveys have gone wrong because of undefined goals or wrong objectives.
For example, if you consider the Net Promoter (NPS) question, you would not survey customers for the sole purpose of knowing IF they would recommend a company or its products. Instead, you should focus on understanding WHY customers would or would not refer.
Also, you want to avoid asking leading questions that influence responder opinions. If you do not avoid this, you can’t obtain “voice of the customer” feedback that is authentic and useful. While these are obvious tips, some things are not as clear, like how much does survey question order matter?
Survey Question Order Tips
When asking responders the NPS question, is it better to ask “how likely are you to recommend” at the beginning or at the end of a survey?
Does the question order create a bias? The jury is still out.
I have tried both ways and believe there are pros and cons to each scenario. For instance, if you ask the Net Promoter question at the end, and the survey is too long, responders may quit. You won’t get the quantitative results needed. On the other hand, if you ask the NPS question first, the answer may not reflect the person’s actual views.
I have been researching the answer to this question in addition to learning from my own survey testing. I like Zontziry (Z) Johnson’s perspective about survey placement and related advantages and disadvantages. In her article, she explains:
The Argument for Placing It First
Putting the overall question first does a few things. First, according to the research, it can actually act as an anchor by which a respondent answers the rest of the questions. So, if we ask an overall question first and get that knee-jerk reaction from a respondent that might be given if asked about their experience by a friend, as they take the rest of the survey, they tend to adjust their following answers to match the answer to the overall question. This is great for the researcher if the answer to that overall question is positive; this isn’t great if that answer is negative.
The argument for placing it last
According to a research article from 2003, “Order Effects in Customer Satisfaction Modelling,” from the Journal of Marketing Management, “…customers’ overall evaluations are more extreme and better explained when provided after attribute evaluations.” By asking the specifics before asking the overall question, respondents will be thinking more about how the specifics add up to their overall level of satisfaction. Arguably, this could mean they are giving a more thought-out answer to that overall question. When should you put the overall question last? When you want your audience to remember various aspects of their experience and give a more thoughtful response. You may also want to consider placing it last if your respondents will be answering awhile after the experience you’re asking them about. That way, you can remind them of the particulars about the experience before asking them to give their overall impression.
Test all Scenarios
What have your experiences with survey design and the order of survey questions been like? Do you ask CX measurement type of questions, like NPS, Satisfaction, and Level of Effort, at the beginning of your survey, at the end or both? Why?
Sign Up for our newsletter to continue learning how to increase your skills and transform your organization! When you register now, you get free access to our whitepaper on how to go from CX Novice to CX Expert.
*All opinions expressed on the DoingCXRight Blog and site pages are the authors’ alone and do not reflect the opinions of or imply the endorsement of employers or other organizations.