How active are you?

In honor of my newest family member Ken, this week’s makeover is for an animal shelter here in Chicago (not Ken’s) that uses a survey to match you with prospective pups.

Apart from the question being a bit on the minimalist side in terms of prompts, an essential aspect of good survey design is this: when someone is reading your question and choosing their answer, you want to make sure that they are understanding the question and choices in the same way as everyone else taking the survey and the way that you (the person who created the survey) thinks they’re going to understand the question.

Huh? You want everyone interpreting your question and answer choices in the same way! Consistency is king.

You won’t definitively know if people are understanding the question how you intend them to, so you want to minimize the risk by being as specific as possible. If everyone reads the question and choices a little differently and then answers based on those differences, well, your data is going to be an unreliable mess.

Let me ask you, are you active or somewhat active?

When you respond, if you say “I’m somewhat active” does that mean more or less active then if you were to say, “I’m active”? For me, I’d think it means less, but based on how the shelter is ordering its response categories, it seems like it means more. Hm. That’s confusing. Because then of course we have “Somewhat Inactive” which seems to be more active than “Inactive” based on the ordering of choices.

If your question sounds like a tongue twister - red flag. Go back to the drawing board, my friend.

This question matters for adoption matches! If people are reading this question and thinking “meh, I’m somewhat active” (meaning a little bit active) and the shelter is thinking that actually means they are more than a little active but less than very active, that is bad news for those adoption matches because ya’ll somewhat active people are going to have dogs that give them a run for their money!

Personally, I’m always wary of questions that have some terms with a descriptor (e.g., Very Active) and some without (e.g., Active). I would rather have everything with a descriptive to make it SUPER clear what I mean. That being said, I can see a descriptorless term being used as a middle category — if you make it clear that it represents a middle!.

Let’s take a look at two alternatives for this question:

You could even imagine options where the shelter gets super specific and asks how many minutes or hours a day you are willing to walk a dog. (Actually, that was one of the questions the shelter asked me when I was interviewing for Ken.) Define what they mean by active — are they curious if you work out a lot or are they wondering if you are someone who walks a dog versus letting them into a backyard a few times a day. This is CRITICAL info for pet owners. Let’s get it right.

Thoughtful survey design is critical. Gosh, this one is going to affect the quality of life for people and animals.