Do you find this sustainability report compelling?

Today’s question makeover is for O My Bag, a fashion brand I love (#notanad) and that also recently released their 2023 Sustainability Report. At first glance — it looks impressive. They even share their actual survey questions (*not the answer choices, though).

It makes me happy to see organizations collecting and using data like this, AND it makes me sad that the questions underlying the data aren’t better. O My Bag is making decisions based on some data that doesn’t tell the whole story. And, that’s bad for sustainability…and business.

I don’t share this to shame O My Bag. I share this to highlight how important it is to spend time designing GOOD surveys.

Below are questions from their Worker Survey Questionnaire:

O My Bag explains the purpose of this survey like this: “Opportunity to voice your opinion every year in the annual worker survey and benefit from the factory improvements.”

I read that, and I think… okay, they are USING this data to improve their factories. That’s important. That makes this data high stakes!

Let’s take a look at the highlighted statements.

Good survey questions only ask about one thing at a time — did you catch the “and” in the first example? Big red flag! “And” signals that the question is double barreled — so, it is asking about two different things at once. This is not good — it makes the question difficult to answer and really difficult to analyze the data. Since O My Bag is using this data to improve their factories, it is critical that they revise this. To improve factories, they need to know if there are issues with ventilation, lighting, or BOTH — this data will not tell them that.

Good survey questions are specific — they don’t leave room for guesswork or misinterpretation — the designer is very clear about what they want to know. Because clarity = good data. In the second question, we have the ambiguous 1 to 10 scale. Gosh, these drive me nuts. If you don’t tell people what a “1” means and what a “10” means, you may find yourself in a situation where some people are answering thinking “1” is the best response and others are answering thinking “10” is the best. You HAVE to define 1 and 10. I have to say, I’m also wondering about the usefulness of this question. Will employees feel comfortable answering this question? What kind of action can O My Bag take based on this question?

Good survey questions are simple — Generally, for each question, I would recommend reading it aloud to see if it reads easily and also flag any tricky words. The goal is to be as simple and straightforward as possible. For O My Bag, this is extra important because these surveys are going out to people in countries where English is not the first language. That makes simple language extra important — workplace, ventilation, and others on the full survey are terms I would recommend revisiting to make the survey as accessible as possible.

Let’s change it up for them:

Can you spot the differences? These new options only ask about one thing at a time and use more simple and straightforward language.

Rather than ask about an overall job rating, I suggest using the Net Promoter question — which is an option for getting at “overall satisfaction” with something (in this case a job) without putting. I could also imagine a scenario where O My Bag asks about specific aspects of the job separately and combines all those questions together to get at “worker satisfaction” — to do this they would think back to their goals: do they care about people having friends at work? having fun at work? feeling like their work is meaningful? feeling appreciated? They could ask about all of these things separately.

What do you think? How would you suggest O My Bag re-write these questions to get them better data so they can improve their factories?

O My Bag wants to improve their factories and that is one of the reasons I LOVE this brand. In order for them to really capture their worker’s experiences and get data that they can use to make improvements — they need to rethink some of these survey questions. Good survey questions get you information that you can trust and that you can use. In order to trust and use the data from a question, you MUST make sure that your question is clear, understandable, and aligns with your goals. Certainly, we could craft a better set of questions to help O My Bag learn more about their factories and how they can improve them. I’d love to!

If you want to get these makeovers and more in real-time, sign up for my weekly newsletter here.