Not all feedback is created e-qual: Why comment boxes are finally useful
Let’s be honest: surveys are tricky, and over time, we (the survey creators) have created some weird habits. We ask people how they feel about something — ranging from a simple online purchase, to something as complex and layered as say, their happiness at work — on a scale of 1 to 10... as if it's that simple!
But if you’ve ever read an open-ended survey comment like “I love your brand but sometimes it feels kinda cheap and the website makes me want to scream into a pillow. 3/5.” you know there’s way more going on beneath the surface.
Luckily, we no longer need to sift through thousands of those kinds of comments with a highlighter and a caffeine dependency (as I have done in the past, at great time-cost). AI is here, and it’s completely changing how we should utilise qualitative feedback.
Quantitative data: Great for telling you what’s wrong, terrible at telling you why
Numbers are fine. They give you structure, trends, and graphs to show in meetings. But they’re also kind of shallow.
Saying, “60% of customers are satisfied” is vague and mysterious. You know full well something’s wrong, but good luck figuring out what.
Qualitative feedback, on the other hand, is where the gold is:
- “The app is okay but crashes every time I try to upload a photo of my cat.”
- “Support was lovely, but I had to go through 6 menus to reach a human being.”
- “Your checkout flow gave me a small existential crisis.”
That’s the stuff you can actually do something with.
I would never want to sit and type what they want, but if someone called me up and said "hey what do you think about xyz in our service?", I could probably talk about it for half an hour. The ‘other comments’ box is where I'd let loose. "
- Quote from a would-be survey respondent
Why AI is a game-changer for qualitative research
Here’s the thing: humans are messy, emotional, inconsistent creatures. And when we type out how we feel, we rarely follow a script.
That’s why AI analysis is such a big deal. It doesn’t just count words, like the traditional, old-school keyword analysis used by many existing tools, it actually understands what people are saying to a surprising degree.
- It can pick up on tone. (Is this “lol” friendly or “lol I hate you” energy?)
- It's excellent at grouping feedback into themes like “users are confused around your pricing” or “shoppers are frustrated with the customer support in your flagship store”, without anyone manually tagging a thing.
- It finds the comments that could either be a red flag… or a brilliant new idea to fix your service.
A real-life example: Picking up nuanced responses from 2051 employees
At Sunbeam, we want to help companies get better through their feedback, and we specialise in creating easy tools to analyse that data.
We recently worked with a global company to survey a sample of their employees - 2,051 of them - to respond to the statement:
"I know the company purpose, and my team and I have taken actions towards becoming more purpose-driven."
The responses were entirely qualitative, in open-ended text boxes.
We approached this with some skepticism, as 'company purpose' can be such an intangible, difficult concept to communicate, and we were also aware of the risk of employees being too subtle with their responses.
However, our analysis found the following:
- Biggest issue: Employees feel disconnected from the company purpose and how to achieve it in their role - 29% of responses
- Biggest positive: There is a general commitment to continuous improvement and team alignment - 14% of responses
- Potential issue: Employees need better communication of the company purpose (and in general) - 5% of responses
- along with a few other themes...
It didn't just pick up "yes/no" responses. A lot of the comments were "yes, but..", or "I don't understand", which came through very clearly.
Quant has its uses, but only if you already know where to look.
Don’t get me wrong, quant data is still useful. It tells you what’s happening at scale, gives you benchmarks, and helps you track trends over time.
But here’s where it doesn't work:
- It assumes you know what to ask. If your survey didn’t include a question about that broken onboarding flow, well… you’re not going to find out.
- It misses emotion. People don’t feel in scores of 1-5. They rant, rave, joke, and confess. That’s the good stuff.
- It’s blind to nuance. As we've just explored in that last example, sometimes people say "yes" when they mean 'yes, but...". Also, a “neutral" score could hide deep annoyance.
If you're a researcher, in customer or user experience, or service design, or if you work in HR - maybe a broad "health check" in the form of a completely open-ended survey would be a great place to start to help you identify some key areas to dive into.
The real deal? Mixing both
Let qualitative feedback tell you where to look, and let numbers give you the scale of the issue.
When you combine the two - quantitative for scale, qualitative for depth - you get insights that are actually worth acting on. And thanks to AI, you can now do it fast, accurately, and without losing your mind.
Now, a basic LLM like ChatGPT is not built for this task. If you were to use, say, our dashboard to analyse your survey results, you know you're using a tool custom-made for the job: consistent, reliable, secure, with the ability to see what your users are asking for, and track changes over a long period of time.
TL;DR
Stop ignoring the comment boxes. In fact, why not lead with them?
Your customers, users, and employees are literally telling you what matters.
So next time someone tells you “the data says people are 76% satisfied,” ask: Yeah, but what did they actually say?
And if you need help with that analysis - let us know
At Sunbeam, we're building a feedback analytics platform primarily focused on the deep analysis of qualitative data. We’re interested in solving the problem of making sense of customer or employee feedback at scale, helping teams cut through the manual work and get straight to the heart of what their feedback is saying.
We're helping companies improve through their feedback. If you're experiencing a challenge like this, or just have a couple of questions - we'd love to talk to you! 😊
