Qualtrics… Are we still using a neutral response category? Really?
You may or may not know that I’m a big fan of Qualtrics — I hate that they raise their prices every year without really making a change in the quality of their service but, alas, I am held captive by the incredible abilities of their platform. I don’t think I can effectively survey without being able to pipe in your previous responses to questions. Seriously. If I ask you what your goals are before we start working together, I want those goals to magically appear in your exit survey. Qualtrics does that.
Qualtrics also loves a neutral response category and ya’ll I just don’t get it, and I am NOT on board.
Is it possible that a neutral response choice could make sense? Yes. Do I often find myself looking at surveys where it makes sense? No.
Below is question from Qualtrics’ recent customer satisfaction survey.
Neither easy nor difficult. Huh? Let’s say, 80% of people picked that category. So, 80% of people said it was neither easy nor difficult to do business with Qualtrics.
What does that mean?
Take a minute and think about it. For you, personally, if a company is neither easy nor difficult to work with… what are they?
Here are the three possibilities, I could come up with:
The neutral option signals that “I don’t know” or “have no opinion”
The neutral option signals that “I don’t feel like answering this question”
The neutral response option means I have an “average” experience with Qualtrics.
Having the neutral suggest an average or typical experience, make a bit of sense; however, if I wanted to know that I would simply ask how your experience with Qualtrics compares to other companies that you do business with because that would be more clear and direct.
If the neutral is a placeholder for not knowing, having no opinion, or generally not feeling like responding? No. Immediately no. That is bad survey design. “Neither easy nor difficult” could mean many things to many people, and that is a clear signal that it is not a quality choice to provide.
What would I suggest instead? Candidly, this one is tricky. Because, to truly craft a good alternative question, I would need to know what Qualtrics actually wants to learn from this question. And, I have no idea. But, here are two possibilities:
Can you see the difference? These reworked questions do a few things.
First, they simplify and get specific about what it means “to do business with.” In this instance, I defined that as setting up or renewing (the language you are shown would depend on whether you are a new or returning client) your license. (which I can assure you is language that the Qualtrics contract holder at your company would be familiar with).
Then, they eliminate the nonsensical neutral response option. In the first re-write, I simply took out the neutral option; in the second, I added a point of comparison which creates an opportunity to say: “hey, my experience with Qualtrics is about the same as other similar companies.”
Good survey questions don’t have answers that leave people saying: huh? Or, that would be difficult for you to explain when you get the data.
