This past Spring, after the Cleveland Cavaliers fell behind 3-0 to the Knicks, a series they'd go on to lose 4-0, head coach Kenny Atkinson told reporters, "Analytically... we have won two out of three."

‍

Analytics matter. They always will. In both sports and market research, data is essential for understanding patterns, probabilities, and performance. But at some point, outcomes matter more than models. If the analytics consistently say you're winning while reality says you're losing, there's a disconnect somewhere; either in the model, the assumptions, the inputs, or the interpretation.

‍

We think market research is entering a similar phase.

‍

The industry is doubling down on synthetic data, synthetic personas, and digital twins, driven by the promise of greater speed and lower cost. Just look at the companies who gets all the speaking slots at industry conferences in the past couple of years. The justification from these companies and case studies increasingly sounds like, "the results are good enough." But "good enough" is a dangerous standard for an industry built on understanding real human behaviour, especially when clients are making high-risk, high-dollar strategic decisions on top of it.

‍

Unbiased research backs up the discomfort. A recent industry presentation on synthetic survey data laid out three findings that are hard to ignore: 1) the concept of synthetic data is logically flawed; 2) it fails empirically to emulate human data; and 3) it cannot overcome sampling limitations. In practice, that shows up as 20-30-point swings between human and synthetic responses in some cases. And this discrepancy is especially big on negative responses, which are often the ones that matter most to a business decision.

‍

We've seen this play out directly. A synthetic panel might return a plausible-sounding reaction to a new product concept, but it won't hesitate the way a real category user does when a claim feels too good to be true, and it won't flag the cultural context that makes a message land differently in one market versus another. Those are the signals that actually change a launch decision, not just a line in a topline report.

‍

The cost of getting this wrong isn't hypothetical. A concept that tests well in a synthetic panel but stalls in market isn't just a missed forecast, it's a real budget spent on the wrong bet, a launch calendar that now has to be rebuilt, and a leadership team that starts questioning the research function itself. Once that trust erodes, it's far harder to rebuild than it would have been to spend the extra two weeks on real fieldwork in the first place.

‍

This matters most in categories where the downside of being wrong is asymmetric: a new product launch, a pricing change, a repositioning that touches years of brand equity. In those cases, the money saved by skipping real fieldwork is a rounding error next to the cost of a launch that underperforms because nobody caught the hesitation in a respondent's voice, or the joke that fell flat with a specific audience segment.

‍

Here's the deeper issue. The most meaningful insights that we have seen in almost 30 years of research have rarely come from clean, perfectly structured responses. They come from hesitation, contradiction, body language, and tone. These are the human signals that reveal what people actually mean, not just what they say. That's what we were trained to listen and watch for. While those signals can be modelled and approximated, they cannot be truly recreated. And that matters, because the job of high-value insights isn't just to produce an answer, it's to persuade and build conviction. Conviction comes from engaging with real people and recognizing something that feels true, not from an output that merely looks right.

‍

This isn't a call to abandon analytics, automation, or AI-assisted tools - they have real value and will keep shaping how we work as an industry. Our view is that this is a call to know where they stop being a tool and start becoming an excuse. As the research and insights world continues to evolve, the advantage won't come from data that's faster, cheaper, or more abundant. It will come from staying close to real people, because that's what earns trust - from your data, and from your clients.

‍

Our own approach reflects this balance. We use AI-assisted tools to speed up the unglamorous parts of the process, transcription, first-pass coding, pattern-spotting across large qualitative datasets, so our researchers and consultants can spend more time where it actually matters: in the room with real people, asking the follow-up to a question a script would have missed. The technology accelerates the process. It doesn't replace the judgment or wisdom.

‍

That's not to say every project needs the same rigor. Early-stage ideation, quick internal pressure-testing, or narrowing a long list of directions before real fieldwork begins are all reasonable places for AI-assisted tools to earn their keep. But the moment a decision carries real budget, reputational, or strategic weight, that's exactly when shortcuts stop paying off, and when the discipline of talking to real people is worth the extra time.

‍

Curious how we uncover the meaningful signals with real human fieldwork? Get in touch.