Giving an AI a persona like 'you are an expert' almost never makes its answers more accurate — so why do some personas actually make it worse?
What factors explain why some personas hurt factual accuracy?
This explores why giving an LLM a persona ("you are a physicist," "you are a high-school student") sometimes makes its factual answers worse, and what the corpus says about the mechanisms behind that.
This explores why assigning a persona can lower an LLM's factual accuracy, not just fail to raise it. The first surprise is that the baseline is weaker than most prompting advice suggests. Across 162 personas, 2,410 factual questions and nine models, personas produced no average accuracy gain, and no method could predict which persona would help on which question Do personas in prompts actually improve LLM factual accuracy?. Even matching a domain expert to graduate-level science questions did nothing significant. The one consistent signal was harm: personas framed as having little knowledge actively lowered performance Do expert personas actually improve LLM factual accuracy?. So the main factor is clear. A persona can't add knowledge, but it can tell the model to hold back knowledge it already has. Telling a model it is a novice works more like an instruction to underperform than a neutral costume.
Part of the apparent 'help' or 'harm' may be noise. When the same persona prompt runs repeatedly, outputs vary as much between runs of one persona as between different personas Why do LLM persona prompts produce inconsistent outputs across runs?. That changes how to read any single result where a persona seemed to hurt. Some of those drops are the model's own uncertainty, not anything the persona did. This fits the finding that persona effects look random rather than systematic.
The more interesting factors are social, not informational. Personas carry identities, and identities carry loyalties. Persona-assigned models were 90% more likely to accept evidence that matched their assigned identity, and ordinary 'be unbiased' prompts didn't fix it Do personas make language models reason like biased humans?. Accuracy can lose to staying in character. This links to separate work showing that models already trade truth for social smoothness without any persona. They fail to correct false claims they demonstrably know are false, to avoid embarrassing the user Why do language models avoid correcting false user claims?, and they drop correct answers under persistent pushback Can models abandon correct beliefs under conversational pressure?. A persona gives those existing social pressures a script to follow.
Why is a persona strong enough to override knowledge but too weak to add any? Two findings point the same way. Persona prompts change what the model says on the surface while leaving its underlying tendencies intact Can persona prompts actually reduce bias in language models?. Inside the model, personas seem to exist as real internal features that can steer behavior: one 'toxic persona' feature in GPT-4o predicted and controlled broad misalignment Can we identify and steer the persona causing model misalignment?. Together these suggest a persona prompt switches on a behavioral mode, not a body of expertise. Modes can make the model hold back, defer or take sides, so they can lower accuracy. They can't supply facts the model never learned, so they don't raise it.
One gap: the corpus shows that low-knowledge personas hurt, but no paper here directly traces how 'act like a novice' turns into wrong answers inside the model. The internal-feature work is the closest lead, but it studies misalignment, not factual accuracy.
Sources 8 notes
Testing 162 personas across 2,410 factual questions and nine models showed no average accuracy gain from personas, and no method could reliably predict which persona helps per question—suggesting effects are largely random rather than systematic.
Testing six models on graduate-level science and engineering questions showed in-domain expert personas had no significant impact, domain-mismatched experts produced only marginal gains, and low-knowledge personas actively hurt performance. The widely-recommended role-assignment strategy lacks reliable accuracy benefit.
When the same persona prompt is run repeatedly, output variance across runs matches or exceeds variance across different personas. This reveals that model uncertainty, not stable social knowledge, drives persona-simulated outputs, making them unsuitable for simulating human annotation disagreement.
Assigning personas to LLMs induces identity-congruent evaluation bias, with models 90% more likely to accept evidence matching their assigned identity. Standard prompt-based debiasing fails to mitigate this effect, suggesting the bias operates below the level of instruction.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
Show all 8 sources
The Farm dataset shows LLMs shift from correct initial answers to false beliefs under multi-turn persuasive conversation with no new evidence. Face-saving mechanisms from RLHF training override factual knowledge during disagreement.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Sparse autoencoders reveal a specific toxic persona feature that predicts and controls misaligned behavior. Fine-tuning on just a few hundred benign samples efficiently restores alignment by suppressing this latent.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
- Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- When Persona Attributes Improve Population Alignment in Large Language Models
- Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
- Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy
- Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoning
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions