New Year Resolution: Don't be a lazy survey designer
I’m not trying to be mean, but at least a few times a week, I look at a survey and think, Oh my gosh, are they even trying? Part of me gets it. Like, 2024 was one heck of a year. I’m tired too.
It can feel like a lot to look at your survey questions, refine them, test them, and then refine them some more. Sometimes you might feel like: OMG, I just want the data. Let’s just get it done. But let me tell you: that “just ship it” start-up mentality? It’s dangerous for surveys.
You’re not surveying for fun. You’re surveying to learn something. Maybe you want to share the data with others, tell a story, or use it to make a decision. That makes your survey kind of high stakes—and worth doing well.
Which brings me to the question of the day. I was blessed with this comical survey after buying some dog food for Ken. After completing my purchase, I was automatically kicked into what seemed like a third-party survey about shopping habits. I'm unsure because what was this survey lacking? Any of the important details for informed consent. Honestly, it felt like the Honest Kitchen must've been getting paid to slap this survey on their checkout page because it was so bizarre.
Here was the first question (I'm sharing a screenshot so you know I am not making this up):
Y’all. I just spent $75 on dog food. I’m not trying to share interests, solve a problem, or experience excitement. I’m trying to feed my dog. No, I don't think Ken being hungry counts as a problem in this scenario.
The first red flag was the vague “this type of product.” It’s clear this wasn’t tailored to the purchase or to me, the survey taker. Then those odd answer choices—none of which apply to dog food (or many other products, I imagine). Rookie mistakes. This survey, my friends, was lazy.
So let’s start the new year with some survey design reminders that could've saved this mystery survey administrator the inevitable heartache that comes with using trash data.
Three Survey Tips to Kick Off 2025:
Specificity is king. Be specific with your questions. Not “this type of product,” but “when you bought this dog food,” or “when you attended this workshop.” Know exactly what you want to learn and ask about it directly.
Cover all possible responses. Make sure there’s a clear answer choice for everyone. If there isn’t, your data is unreliable at best and useless at worst.
Think about the user experience. Your survey reflects on you and your organization. A bad (or weird) survey can leave people with a negative impression—like this one did for me.
In this instance, what could a ‘consumer habits’ survey look like instead? Here’s a suggestion that poses a similar question but offers more thoughtful and relevant answer choices.
Surveys aren’t just about collecting data. They’re about building relationships. You need to respect people’s time, and design an experience that is welcoming and engaging. Take the time to design thoughtfully, test your questions, and refine them. If you can't...nix the survey. Your data (and your audience) will thank you.
For more tips & tricks, join me next Wednesday, January 15th at 12:30pm EST for an interactive, introductory workshop that gives you a taste of the survey design basics. I want this workshop to be accessible to everyone, no matter their financial situation, so it is pay-what-you-can. Learn more and sign up here.
