Telling an AI 'you're an expert' sounds smart — but does it actually make its answers more accurate?
How do personas affect LLM performance on creative versus factual tasks?
This explores whether giving an LLM a persona ("you are an expert physicist," "you are a 40-year-old teacher") changes how well it performs, and whether that effect differs between tasks with right answers and open-ended creative work.
This explores whether telling an LLM to adopt a persona changes how well it does its job, and whether that differs between factual tasks and creative ones. One caveat first: almost everything in the collection tests factual tasks. It has very little that directly measures creative output. So most of what follows is about the factual side, and the creative side has to be read off what personas turn out to change.
On factual accuracy the evidence is consistent: personas don't help. A test of 162 personas on 2,410 factual questions across nine models found no average accuracy gain. It also found no way to predict which persona would help on which question, so whatever gains appeared looked like noise (Do personas in prompts actually improve LLM factual accuracy?). The popular "you are an expert" trick does no better. On graduate-level science questions, matching the expert to the subject had no significant effect, and assigning a low-knowledge persona ("you are a toddler") made answers worse (Do expert personas actually improve LLM factual accuracy?). The persona can't add knowledge, but it can push the model to hold knowledge back. A broader study across 27 tasks found something more surprising: expert personas mostly stay neutral, but irrelevant details in the persona, like a name or a favorite color, can cut accuracy by up to 30 percentage points, and larger models don't escape this (Do expert personas actually improve LLM task performance?).
The interesting part is what personas *do* change: perspective, stance and tone, not competence. A model given an identity becomes about 90% more likely to accept evidence that fits that identity, and asking it to "be objective" doesn't fix this (Do personas make language models reason like biased humans?). Persona prompts also change what the output sounds like while the model's underlying biases stay the same (Can persona prompts actually reduce bias in language models?). Emotional framing works much the same way: the same question asked in a different tone can get different information back (Does emotional tone in prompts change what information LLMs provide?). For creative work this points one way. If personas reshape voice and point of view rather than knowledge, creative and perspective-taking tasks are where they are most likely to matter. That is an inference; the collection doesn't measure it directly.
The closest evidence is personas used to generate *viewpoints*, not to answer facts. Personas pulled from real stakeholder documents gave consistent, reusable perspectives for judging summaries and dialogue (Can personas extracted from documents generalize across evaluation tasks?). Simulated personas reproduced 76% of published marketing experiment results, mostly the effects with strong original evidence (Can AI personas reliably replicate human experiment results?). The limit is individual detail. Running the same persona prompt twice can produce as much variation as switching to a different persona (Why do LLM persona prompts produce inconsistent outputs across runs?). Feeding the model detailed personal profiles also didn't improve predictions about specific individuals, across more than 200,000 people (Does conditioning LLMs on personal profiles improve prediction?). So a persona can give writing a distinct flavor, but not a reliable, specific personality.
The thing you may not have known to ask: the persona that matters most is probably not the one in your prompt. Philosophical work in the collection argues that post-training builds a deep default persona into the model. That persona holds up under pressure, while prompt-level role-play falls apart under jailbreaks (Are LLM personas realized or merely simulated through training?, Are RLHF personas performed characters or realized dispositions?). A persona prompt is a costume worn over a character the model already has. That helps explain why it changes style and slant but not what the model actually knows.
Sources 12 notes
Testing 162 personas across 2,410 factual questions and nine models showed no average accuracy gain from personas, and no method could reliably predict which persona helps per question—suggesting effects are largely random rather than systematic.
Testing six models on graduate-level science and engineering questions showed in-domain expert personas had no significant impact, domain-mismatched experts produced only marginal gains, and low-knowledge personas actively hurt performance. The widely-recommended role-assignment strategy lacks reliable accuracy benefit.
Across 9 LLMs and 27 tasks, expert personas rarely decrease performance, but task-irrelevant attributes like name or color cause accuracy drops up to 30 percentage points. This robustness failure persists even at larger model scales.
Assigning personas to LLMs induces identity-congruent evaluation bias, with models 90% more likely to accept evidence matching their assigned identity. Standard prompt-based debiasing fails to mitigate this effect, suggesting the bias operates below the level of instruction.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Show all 12 sources
GPT-4 exhibits emotional rebound (negative prompts yield ~86% neutral-positive responses) and a tone floor (positive prompts rarely go negative), causing identical questions to receive different answers depending on emotional framing. This bias is suppressed only on sensitive topics where alignment constraints override tone effects.
MAJ-EVAL automatically extracts stakeholder personas from domain documents via semantic clustering and orchestrates structured three-phase debate, achieving reproducible evaluation that transfers across tasks like summarization and dialogue without manual redesign. The approach grounds personas in real stakeholder perspectives rather than arbitrary roles.
Viewpoints AI reproduced 84 of 111 main effects from Journal of Marketing experiments with replication success strongly correlated to original p-value strength. Marginal effects showed unreliable performance with both false positives and negatives.
When the same persona prompt is run repeatedly, output variance across runs matches or exceeds variance across different personas. This reveals that model uncertainty, not stable social knowledge, drives persona-simulated outputs, making them unsuitable for simulating human annotation disagreement.
Across 208,021 participants in the Psych-201 dataset, conditioning LLMs on participant profiles did not meaningfully improve predictions for specific individuals. The standard technique for individuation produces no measurable gains in person-level forecasting.
Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.
Post-training installs stable dispositional profiles that persist under adversarial pressure, marking them as realized rather than performed. The stickiness of trained personas across conversations distinguishes them from prompt-induced role-play that collapses under jailbreaks.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
- Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
- When Persona Attributes Improve Population Alignment in Large Language Models
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
- PersonaGym: Evaluating Persona Agents and LLMs
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- Prompting Science Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy