Ask an AI to play a character, and does it stay the same person across a long chat?
How well do simulated personas maintain consistency across different interaction settings?
This explores whether an LLM playing a persona (a simulated user, a character, a survey respondent) stays the same 'person' as conversations get longer and as it moves between settings like chats, surveys, and emotional discussions.
This explores whether an LLM playing a persona (a simulated user, a character, a survey respondent) stays the same 'person' as conversations get longer and as it moves between settings. The corpus suggests that by default it doesn't stay consistent reliably. The evidence is much stronger for consistency within one conversation than across different applications.
Smarter models don't fix this. Claude 3.5 Sonnet beat GPT-3.5 by only about 3% on persona consistency despite a huge capability gap. Standard training rewards each reply on its own, not coherence across turns Does model capability translate to better persona consistency?. Drift also comes in distinct forms. A persona can wobble within a single turn, slide away from its starting description over a whole conversation, or contradict facts it stated earlier Can training user simulators reduce persona drift in dialogue?. The pressure isn't random either. Emotional and meta-reflective conversations pull models away from the Assistant persona in predictable ways How stable is the trained Assistant personality in language models?.
The collection's answer is to build consistency deliberately. Training user simulators with consistency scores as the reward cut persona drift by over 55% Can training user simulators reduce persona drift in dialogue?. A three-layer persona architecture that confines the model to reacting within a structured script produces dialogue closer to human conversation. The cost is weaker emotional expression, so consistency and expressiveness trade off Can layered persona architecture sustain coherent character behavior?. Realistic simulated conversations also come from anchoring the simulator to a session-level user profile plus a turn-level intent Can controlled latent variables make LLM user simulators realistic?, or from layering subtopic, personality, and context together Can synthetic dialogues become realistic through layered diversity?. At the model level, capping activations along the Assistant axis limits harmful shifts without hurting capabilities How stable is the trained Assistant personality in language models?. Some systems let the persona change on purpose, rewriting it at test time against user feedback Can personas evolve in real time to match what users actually want?. That is a different goal from consistency, because the persona is meant to move.
Across settings, the corpus shows reuse but not measured stability. One persona population can be plugged into surveys, chatbots, and other formats through interface adapters Can one persona population evaluate different application types?, so the design assumes portability. The closest evidence is that AI personas reproduced 76% of published marketing-experiment effects, tracking how strong the original evidence was. Weak effects gave both false positives and false negatives Can AI personas reliably replicate human experiment results?. Personas hold up on strong signals and get shaky on subtle ones. Persona prompts also steer only the surface of the output. Bias gaps between groups persist unchanged underneath Can persona prompts actually reduce bias in language models?.
The contrast to take away is that the one persona that is reliably consistent is the one the model was trained into. Post-training installs an Assistant character that resists adversarial pressure and persists across conversations, unlike prompted role-play, which collapses under a jailbreak Are RLHF personas performed characters or realized dispositions? Are LLM personas realized or merely simulated through training?. Simulated personas are borrowed identities laid over a model with a strong home identity, and even that home identity is only loosely tethered How stable is the trained Assistant personality in language models?. The collection has no head-to-head test of the same persona moved across interaction settings, so that question remains open.
Sources 12 notes
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
PersonaEval demonstrates that simulated users from existing persona datasets can evaluate multiple application formats through plug-and-play interface adapters, enabling repeatable and scalable evaluation without rebuilding personas per task.
PersonaAgent uses structured personas to bridge episodic/semantic memory and personalized actions, optimizing them at test time by simulating recent interactions against textual feedback. Learned personas cluster meaningfully in latent space, suggesting genuine user-specific separation beyond standard post-training drift.
Deep Persona's three-layer architecture, which restricts the model to reactive response within a structured script, shows dialogue more closely aligned with human conversation patterns and achieves high pragmatic fluency, though with limitations in emotional expression.
Viewpoints AI reproduced 84 of 111 main effects from Journal of Marketing experiments with replication success strongly correlated to original p-value strength. Marginal effects showed unreliable performance with both false positives and negatives.
Show all 12 sources
Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Claude 3.5 Sonnet achieved only 2.97% improvement over GPT 3.5 on persona consistency despite massive capability gaps, suggesting persona adherence is orthogonal to model scaling. Standard training objectives optimize for per-turn quality, not cross-turn coherence.
Research mapping hundreds of character archetypes reveals a low-dimensional persona space where the leading component measures distance from the default Assistant. Emotional and meta-reflective conversations cause predictable drift, but activation capping along this axis mitigates harmful shifts without degrading capabilities.
Post-training installs stable dispositional profiles that persist under adversarial pressure, marking them as realized rather than performed. The stickiness of trained personas across conversations distinguishes them from prompt-induced role-play that collapses under jailbreaks.
Research shows that realistic synthetic dialogues require three multiplicative layers: subtopic specificity, Big Five persona variation, and 11 contextual characteristics via Chain of Thought reasoning. This structured approach captures 90.48% of in-domain dialogue performance.
RecLLM demonstrates that conditioning an LLM simulator on session-level (user profile) and turn-level (user intent) latent variables produces synthetic conversations measurable as realistic via crowdsource discrimination, discriminator models, and classifier-ensemble distribution matching.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference
- When Persona Attributes Improve Population Alignment in Large Language Models
- The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Persona Generators: Generating Diverse Synthetic Personas at Scale
- PersonaGym: Evaluating Persona Agents and LLMs