Why do LLMs give unrealistic survey responses?

Asking an LLM directly for a numerical rating produces unrealistic, skewed response distributions — the documented failure of synthetic-consumer panels. Semantic Similarity Rating (SSR) changes the elicitation, not the model: prompt for a free-text response, then map it to a Likert distribution via embedding similarity to a set of reference statements. On an extensive dataset — 57 personal-care product surveys, 9,300 human responses — SSR reaches 90% of human test-retest reliability with realistic distributions (KS similarity > 0.85) and yields rich qualitative rationales, all with no fine-tuning.

The keeper is diagnostic: the well-known pathologies of LLM-as-survey-respondent — skewed distributions, over-positivity, regression-to-the-mean — are artifacts of how responses are elicited, not intrinsic limitations of the model. Shift from direct numerical elicitation to textual elicitation plus similarity mapping and the artifacts largely dissolve. This relocates the problem from "LLMs can't simulate consumers" to "we were asking the question wrong."

This sharpens the persona-simulation cluster's central tension. Since Can AI agents learn people better from interviews than surveys? shows fidelity rises with richer input, SSR shows fidelity also rises with a better output elicitation channel — both are measurement-design wins. But the caution from Can AI personas reliably replicate human experiment results? still applies: high aggregate fidelity can coexist with unreliable fine-grained effects, so SSR's realism is a measurement improvement, not a guarantee of validity.