Synthetic audiences are everywhere now, but the question that actually decides whether one is worth trusting is rarely asked: how much of a real person does the model need before it behaves like the original?

In a Columbia study, more than two thousand people each answered five hundred questions about themselves and an AI twin was built of every one, and the twins reproduced people’s held-out answers at about eighty-eight percent of the rate the humans matched their own answers two weeks later.

The striking part: more than a dozen ways of building the twin — different models, formats, prompting tricks, even fine-tuning — barely changed the result. What carried it was the depth of the data, not the cleverness of the model.

The same lesson shows up when Bain backtested synthetic audiences against a company’s real prior study and concluded the data grounding a model matters more than the model itself, and when BCG reported a synthetic panel predicting real shoppers’ choices with ninety-two percent accuracy — after fine-tuning on real data. A working paper from University College Dublin adds the other half: demographics alone can’t capture how a real person decides, and the data has to be organized for the job it’s doing.

The takeaway for brand leaders: when someone hands you a customer model, stop asking which AI is under the hood and start asking how much real, first-party data it’s built on — and whether it was tuned for the decision you’re making.

Large language models are now embedded in how many brands do market research — simulate consumer responses, get results in hours instead of weeks, at a fraction of the cost.

But a rigorous Harvard Business School and Microsoft Research working paper finds that LLM-based preference estimates are realistic in some contexts and wrong-signed in others — with estimates sometimes off by a factor of three — and that a researcher without a human benchmark has no basis for knowing which is which. A Journal of Marketing study finds the human-LLM hybrid outperforms both approaches alone, but only when the human judgment is genuine rather than a pass-through. And research published in Harvard Business Review finds that LLMs consistently recommend strategies aligned with managerial buzzwords rather than context-specific logic — a pattern the researchers named trendslop.

Taken together, the evidence points to a specific map: LLMs for market research are reliable supplements when you have prior human data from the same category, unreliable substitutes when you don’t, and systematically biased toward the fashionable on exactly the questions where conventional industry thinking most needs challenging.

This episode makes that map practical — three questions a brand leader should ask before acting on what AI-assisted research tells them.

Synthetic respondents have gone mainstream — AI stand-ins for your customers you can question without recruiting anyone, a focus group that never closes.

But not all of them are worth listening to, and this episode is about what actually decides that: not the fact that they never end, but the depth and realness of the data the model is built on, and whether it was built for the job you’re asking it to do.

We draw on a Stanford study where AI versions of real people, built from rich interviews, far outperformed versions built from demographics alone — and on its 2026 follow-up breaking the result down by how much real data each model was given — alongside new work finding that piling demographic detail onto a generic model doesn’t reliably make it more accurate, to get specific about what these modelled respondents are, why a stand-in is only ever as good as the data underneath it, and the two questions a brand leader should ask before trusting one — what is it built on, and what was it built for?

A model grounded in deep, real customer data is an instrument; a thin guess in a persuasive font is a mirror.