When an AI plays a person, how can you tell it thinks like a human rather than just sounding like one?

This explores how researchers can tell whether a simulated person or persona behaves like a real human mind, and not just sounds like one. The corpus has no single accepted test for this. It has at least four different yardsticks, and each catches failures the others miss. The first is whether the simulation reproduces real experimental results. One persona system reproduced 84 of 111 main effects from published marketing experiments (76 percent), and its success tracked how strong the original evidence was. It was unreliable on marginal effects, with both false positives and false negatives (Can AI personas reliably replicate human experiment results?). Outcome replication is a useful headline number, but it tells you little about the weak-signal cases where you'd most want to trust a simulation.

The second yardstick is whether anyone can tell the simulation from the real thing. RecLLM's user simulator was conditioned on a user profile for the whole session and an intent for each turn. Its realism was then checked three ways: crowdworkers trying to spot fakes, a trained discriminator model, and a classifier ensemble comparing the distribution of simulated conversations against real ones (Can controlled latent variables make LLM user simulators realistic?). The third is whether the persona stays itself over time. Drift can be scored at three levels: a line that contradicts the persona prompt, a line that contradicts earlier lines, and factual contradictions when the persona is quizzed. Using those scores as a training reward cut drift by more than 55 percent (Can training user simulators reduce persona drift in dialogue?).

Static tests can be passed by a shallow imitation, so the corpus pushes toward harder probes. Language models do well on structured mind-reading tasks but default to surface strategies in open-ended perspective-taking. Forcing explicit belief tracking helped, which suggests the gap is architectural (Do large language models genuinely simulate mental states?). Personality-conditioned agents are also tested on change. After life events their traits do shift, but the shifts are generic, weakly tied to the specific event, smaller than human effects, and squashed together across personas (Do personality-conditioned agents change like humans do?). An agent can look psychologically realistic in a snapshot and still lack a realistic developmental trajectory.

A realism test also has to include the unflattering parts of psychology. On the Moral RolePlay benchmark, scores fall from 3.21 for moral paragons to 2.62 for villains. Models fail most on deception and manipulation, replacing nuanced malevolence with crude aggression (Does safety alignment harm models' ability to roleplay villains?). A simulator validated only on agreeable characters will overstate how human it is. Another approach asks whether a persona is stable enough to count as real. On the realizationist account, post-training installs dispositions that survive adversarial pressure, while prompt-induced role-play collapses under jailbreaks (Are RLHF personas performed characters or realized dispositions?, Are LLM personas realized or merely simulated through training?). That is a measurable robustness test. It measures whether the persona is genuine, though, not whether it is human-like. The role-play framing keeps the distinction clear by applying folk psychology to the character and not to the system underneath (Should we treat dialogue agents as role-playing characters?).

There are two cautions about what you measure. If you ask people to rate realism, competence dominates their impression of a dialogue partner (49 percent of the variance), well ahead of human-likeness (32 percent) (How do users mentally model dialogue agent partners?). Rating-based realism scores may therefore reward capable agents over human-like ones. Benchmarks also shape what gets built. Agents that win abstract contests fail on long-horizon real work, because the field optimized what it measured (Why do agent benchmarks not predict real economic value?). The practical takeaway is to combine tests: replication of real results, distinguishability from real data, consistency across a conversation, open-ended mental-state probes, dynamics over time, and dark-trait fidelity. Report them separately, because a simulator can pass some and fail others.