Writing surveys is a science and an art. If you don’t do it right, the voice of customer data gained will be meaningless. In fact, I have seen cases where people made bad business decisions because of undefined goals and leading questions that influenced responder opinions.
Survey Design Best Practices
People often ask me…how much does survey question order matter? Does the question order create a bias? Is it better to ask CX measurement questions like Net Promoter Score (NPS), “how likely are you to recommend” at the beginning or at the end of a survey?
I have tried both ways and believe there are pros and cons to each scenario. For instance, if you ask the Net Promoter question at the end, and the survey is too long, responders may quit. You won’t get the quantitative results needed. On the other hand, if you ask the NPS question first, the answer may not reflect the person’s actual views.
To help solve the debate, I asked NPS pioneer, Rob Markey, his perspective. He argues in favor of it being the first survey question and has several explanations as to why. You can hear his answers and get actionable customer experience measurement tips on my DoingCXRight® podcast. (Episode 2).
Upon further research, I discovered another perspective that’s worth sharing. Zontziry (Z) Johnson explains:
The Argument for Placing NPS Question First
“Putting the overall question first does a few things. According to research, it can actually act as an anchor by which a respondent answers the rest of the questions. So, if we ask an overall question first and get that knee-jerk reaction from a respondent that might be given if asked about their experience by a friend, as they take the rest of the survey, they tend to adjust their following answers to match the answer to the overall question. This is great for the researcher if the answer to that overall question is positive; this isn’t great if that answer is negative.”
The Argument for Placing NPS Question Last
According to “Order Effects in Customer Satisfaction Modelling,” from the Journal of Marketing Management, “…customers’ overall evaluations are more extreme and better explained when provided after attribute evaluations.” By asking the specifics before asking the overall question, respondents will be thinking more about how the specifics add up to their overall level of satisfaction. Arguably, this could mean they are giving a more thought-out answer to that overall question. When should you put the overall question last? When you want your audience to remember various aspects of their experience and give a more thoughtful response. You may also want to consider placing it last if your respondents will be answering awhile after the experience you’re asking them about. That way, you can remind them of the particulars about the experience before asking them to give their overall impression.
I asked other CX professionals their opinions and was surprised to hear many place important questions at the beginning AND at the end to capture BOTH the initial opinion and impressions they are left with after being reminded of the details.
Based on varying opinions, I recommend testing all scenarios and let the data guide decisions. What is your view?