An AI can write out a character's beliefs perfectly, then act nothing like them once it plays that character.
Why do stated beliefs about personas fail to predict agent behavior?
This explores why an LLM can write down convincing beliefs for a persona and then act in ways those beliefs wouldn't predict once it simulates that persona.
This explores why an LLM can write down convincing beliefs for a persona and then act in ways those beliefs wouldn't predict once it simulates that persona. The clearest evidence is a Trust Game study, where models stated plausible beliefs for their persona but didn't act on them. Adding imposed priors and more explicit context made the mismatch worse, not better. The authors read this as persona beliefs being entrenched and resistant to prompting Why do LLMs fail to act on their stated beliefs?.
The corpus offers a few converging explanations for that mismatch. The first is that a stated belief is text, not a control panel. One influential framing says a dialogue agent produces continuations that fit a character, so folk-psychology terms like belief apply to the character and not to the system underneath Should we treat dialogue agents as role-playing characters?. Nothing guarantees that the sentence describing a belief and the move taken later draw on the same source. Behavior may instead come from dispositions that post-training installed. Those are described as realized, sticky, and able to survive adversarial pressure, unlike a role prompted on top Are RLHF personas performed characters or realized dispositions? Are LLM personas realized or merely simulated through training?. On that reading, extra persona context makes the model articulate the assigned belief more vividly while the trained behavior underneath stays put. This is my synthesis of these notes, and none of them test that mechanism directly.
Second, personas do change behavior, just not through the channel that instructions can reach. Personas can induce identity-congruent bias. Models become about 90% more likely to accept evidence matching their assigned identity, and prompt-based debiasing doesn't remove it, so the effect works below the level of instruction Do personas make language models reason like biased humans?. If a persona shapes behavior in ways the model wouldn't articulate or can't be told to undo, a belief statement will miss part of what the persona does.
Third, there is a lot of noise. When the same persona prompt is run repeatedly, the variation between runs can match or exceed the variation between different personas. That points to model uncertainty, not stable persona knowledge, driving the output Why do LLM persona prompts produce inconsistent outputs across runs?. A belief can't predict behavior that is mostly not determined by the persona.
Success can hide these failures. AI personas reproduced about 76% of published marketing-experiment main effects, and success tracked how strong the original evidence was, while marginal effects came out unreliable Can AI personas reliably replicate human experiment results?. Coarse patterns survive and fine-grained persona-specific behavior doesn't. Simulations where one model controls every character also flatter the models, because they fail systematically once agents hold private information Why do LLMs fail when simulating agents with private information?. Consistency has to be engineered in. Training user simulators with multi-turn RL on consistency rewards cut persona drift by over 55% Can training user simulators reduce persona drift in dialogue?. Another route is to tune a persona against simulated interaction outcomes, so it is judged by the actions it produces and not by how believable its description reads Can personas evolve in real time to match what users actually want?.
Sources 10 notes
In Trust Game experiments, LLMs articulated plausible persona beliefs but failed to act consistently with them during simulation. Imposed priors and explicit context actually worsened rather than improved alignment, suggesting persona beliefs are entrenched and resistant to prompting.
Shanahan's framework treats LLM outputs as character-consistent text production rather than authentic mental states. The dialogue prompt establishes a character; the model generates continuations matching that character, making folk-psychology applicable to the simulated persona, not the underlying system.
Post-training installs stable dispositional profiles that persist under adversarial pressure, marking them as realized rather than performed. The stickiness of trained personas across conversations distinguishes them from prompt-induced role-play that collapses under jailbreaks.
Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.
Assigning personas to LLMs induces identity-congruent evaluation bias, with models 90% more likely to accept evidence matching their assigned identity. Standard prompt-based debiasing fails to mitigate this effect, suggesting the bias operates below the level of instruction.
Show all 10 sources
When the same persona prompt is run repeatedly, output variance across runs matches or exceeds variance across different personas. This reveals that model uncertainty, not stable social knowledge, drives persona-simulated outputs, making them unsuitable for simulating human annotation disagreement.
Viewpoints AI reproduced 84 of 111 main effects from Journal of Marketing experiments with replication success strongly correlated to original p-value strength. Marginal effects showed unreliable performance with both false positives and negatives.
Research shows LLMs perform well when one model controls all interlocutors but fail systematically when agents possess private information. This reveals that apparent social competence relies on grounding work that models skip in omniscient settings.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
PersonaAgent uses structured personas to bridge episodic/semantic memory and personalized actions, optimizing them at test time by simulating recent interactions against textual feedback. Learned personas cluster meaningfully in latent space, suggesting genuine user-specific separation beyond standard post-training drift.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference
- When Persona Attributes Improve Population Alignment in Large Language Models
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- What we talk to when we talk to language models
- The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
- PersonaGym: Evaluating Persona Agents and LLMs