When you tell an AI something new about yourself, does it update its picture of you, or just make things up?
How do persona signals change when users provide new evidence about themselves?
This explores what happens to an AI's working picture of a user (its persona signals) when the user reveals something new about themselves: whether the picture gets properly revised, embellished beyond the evidence, or pushed around by the conversation itself.
This explores what happens to an AI's working picture of a user when the user reveals something new about themselves: whether it gets properly revised, embellished beyond the evidence, or pushed around by the conversation. The corpus has no single study of that exact moment. It does have pieces that fit together, and they point to one idea: a persona signal should be treated as a claim that evidence can support or overturn, not as a fixed label.
The most direct piece is PGMem. Flat persona profiles, such as a summary that says "likes hiking, vegetarian", have no record of why the system believes something, so new evidence can only be pasted on or silently overwrite the old line. PGMem instead links each persona signal to the events that support or revise it through typed evidence edges. When a user says something new, it becomes another event attached to the relevant signal, and the signal can be re-examined against its evidence. On small language models this beat summary-based and graph-structured baselines, and the gains grew with context size (Can persona signals stay grounded in their supporting events?). PersonaAgent takes a complementary route: it treats the persona as a live bridge between memory and action, then rewrites it at test time by simulating recent interactions against feedback. The learned personas cluster by user in latent space, which suggests they capture something user-specific (Can personas evolve in real time to match what users actually want?).
The risk is that models don't stop at what the user said. MirageBench found that all 12 tested LLMs over-infer 35 to 49 percent of their claims about a user, driven by verbosity, pretraining priors, and genre expectations. Models that rated themselves as over-inferring less actually over-inferred more when judged independently (Do large language models fabricate user attributes beyond available evidence?). So a small new disclosure can be inflated into a confident portrait, and the model can't reliably tell it has done so. This is the strongest argument for keeping signals traceable to evidence: it lets you check whether a claim about a user is grounded.
A second problem is that persona change isn't always caused by evidence. In user simulators, personas drift within a turn, across a whole conversation, and into factual contradictions. Training explicitly for consistency cut that drift by over 55 percent (Can training user simulators reduce persona drift in dialogue?). The assistant's own persona also slides away from its default in emotional or meta-reflective conversations. Those are the very conversations where users tend to share personal details (How stable is the trained Assistant personality in language models?). Persona-prompted outputs can also vary as much from run to run as between different personas, because model uncertainty rather than stable knowledge is driving them (Why do LLM persona prompts produce inconsistent outputs across runs?). A shift after new evidence might therefore be a real update or just noise.
How much new evidence should move a persona depends on how strong the signal is. Personas built from real behavioral data predicted A/B test direction well for large effects and poorly for near-zero ones (Can behavior-based personas predict A/B test outcomes?). Simulated personas replicated published experiments in proportion to the original evidence strength, and were unreliable on marginal effects (Can AI personas reliably replicate human experiment results?). Weak evidence about a user gives a weak signal, and a system that treats it as strong will overreact. Adding a fact to a prompt can also change only the surface. Persona prompting changes how outputs sound while leaving underlying bias in place (Can persona prompts actually reduce bias in language models?). The likely pattern is that the model changes its wording after new evidence without changing what it actually infers about you.
Sources 9 notes
PGMem addresses validity and retrieval gaps in flat persona profiles by connecting persona signals to events through typed evidence edges. Testing on small language models showed outperformance over summary-based and graph-structured baselines, with gains that grow with context size.
PersonaAgent uses structured personas to bridge episodic/semantic memory and personalized actions, optimizing them at test time by simulating recent interactions against textual feedback. Learned personas cluster meaningfully in latent space, suggesting genuine user-specific separation beyond standard post-training drift.
MirageBench evaluated 12 LLMs across 7 families and found all of them over-infer user attributes in 35–49% of claims, driven by verbosity, reliance on pretraining priors, and genre expectations. Models that self-assess as over-inferring less actually over-infer more when judged independently.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
Research mapping hundreds of character archetypes reveals a low-dimensional persona space where the leading component measures distance from the default Assistant. Emotional and meta-reflective conversations cause predictable drift, but activation capping along this axis mitigates harmful shifts without degrading capabilities.
Show all 9 sources
When the same persona prompt is run repeatedly, output variance across runs matches or exceeds variance across different personas. This reveals that model uncertainty, not stable social knowledge, drives persona-simulated outputs, making them unsuitable for simulating human annotation disagreement.
LLM agents conditioned on anonymized behavioral data predicted A/B test directions with 0.75–0.90 accuracy across 40 experiments. Predictions were most reliable for large effects and least trustworthy for near-zero effects, making the approach viable for fast pre-screening but not full replacement of live testing.
Viewpoints AI reproduced 84 of 111 main effects from Journal of Marketing experiments with replication success strongly correlated to original p-value strength. Marginal effects showed unreliable performance with both false positives and negatives.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- When Persona Attributes Improve Population Alignment in Large Language Models
- Persona Generators: Generating Diverse Synthetic Personas at Scale
- The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
- Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation
- PersonaEval: Persona-Based User Simulation for Evaluating Interactive Applications