Tell an AI it's an expert and it performs the same — but tell it it's a novice, and it actually gets dumber.
Do persona effects depend on the knowledge level of the role?
This explores whether telling an AI to play a high-knowledge role (an expert) versus a low-knowledge role (a novice or layperson) changes how well it actually performs, and whether the 'role' changes what the model knows or only how it presents what it knows.
This explores whether a role's knowledge level, expert versus novice, changes how well an AI performs, or only how it sounds. The corpus says yes, it matters, but in only one direction. When six models answered graduate-level science and engineering questions, giving them a matching expert persona ("you are a physicist") made no significant difference. A mismatched expert gave only marginal gains. A low-knowledge persona ("you are a child," "you are a layperson") actively made answers worse Do expert personas actually improve LLM factual accuracy?. A persona can take capability away, but it can't add any. The model already 'knows' what it knows, and naming a role doesn't unlock anything new. Telling it to act less informed, though, it will happily do.
The asymmetry makes sense once you see where persona prompts act. Research on bias found that personas shape the surface of the output: the model follows the trait instructions, but the underlying patterns stay the same Can persona prompts actually reduce bias in language models?. Apply that to knowledge. An expert label can't deposit expertise below the surface, but a novice label gives the model permission to hold back, simplify, or get things wrong on purpose. Performing ignorance is easy. Performing knowledge you don't have isn't possible.
The same pattern appears in very different tasks. A plain 'typical reader' prompt ranked real headline variants against actual click data better than a panel of ten demographic personas Do demographic personas help models rank headlines better?. Across more than 200,000 participants, conditioning models on personal profiles didn't improve predictions about specific individuals Does conditioning LLMs on personal profiles improve prediction?. In both cases the persona added framing and bias, not insight. Even staying in character turns out to be its own skill, unrelated to general intelligence: a far more capable model kept a persona only about 3% more consistently than a much older one Does model capability translate to better persona consistency?.
There is a twist. Personas installed through training, rather than written into a prompt, seem to behave differently. They persist across conversations and resist adversarial pressure, which suggests they are built-in dispositions rather than performances Are RLHF personas performed characters or realized dispositions? Are LLM personas realized or merely simulated through training?. So 'does knowledge level matter?' may have a different answer depending on whether the role comes from a prompt or from training. The corpus doesn't yet test expert versus novice roles for trained personas, and that gap is worth noticing.
The practical takeaway: drop the 'act as an expert' prompt for accuracy, because it's mostly ceremony. Watch out for any role that implies less knowledge, such as a student, a beginner, or a specific demographic profile, because that is where personas quietly cost you performance. Personas earn their keep elsewhere: grounding evaluators in real stakeholder perspectives drawn from documents Can personas extracted from documents generalize across evaluation tasks?, or simulating users for testing Can one persona population evaluate different application types?. There, the point is to represent a viewpoint, not to raise competence.
Sources 9 notes
Testing six models on graduate-level science and engineering questions showed in-domain expert personas had no significant impact, domain-mismatched experts produced only marginal gains, and low-knowledge personas actively hurt performance. The widely-recommended role-assignment strategy lacks reliable accuracy benefit.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
A zero-shot baseline asking how likely a typical reader is to click ranked Upworthy headline variants significantly better than a ten-persona demographic panel. Persona conditioning appears to introduce systematic bias rather than capture audience-specific insight.
Across 208,021 participants in the Psych-201 dataset, conditioning LLMs on participant profiles did not meaningfully improve predictions for specific individuals. The standard technique for individuation produces no measurable gains in person-level forecasting.
Claude 3.5 Sonnet achieved only 2.97% improvement over GPT 3.5 on persona consistency despite massive capability gaps, suggesting persona adherence is orthogonal to model scaling. Standard training objectives optimize for per-turn quality, not cross-turn coherence.
Show all 9 sources
Post-training installs stable dispositional profiles that persist under adversarial pressure, marking them as realized rather than performed. The stickiness of trained personas across conversations distinguishes them from prompt-induced role-play that collapses under jailbreaks.
Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.
MAJ-EVAL automatically extracts stakeholder personas from domain documents via semantic clustering and orchestrates structured three-phase debate, achieving reproducible evaluation that transfers across tasks like summarization and dialogue without manual redesign. The approach grounds personas in real stakeholder perspectives rather than arbitrary roles.
PersonaEval demonstrates that simulated users from existing persona datasets can evaluate multiple application formats through plug-and-play interface adapters, enabling repeatable and scalable evaluation without rebuilding personas per task.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
- When Persona Attributes Improve Population Alignment in Large Language Models
- Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
- PersonaGym: Evaluating Persona Agents and LLMs
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Persona Generators: Generating Diverse Synthetic Personas at Scale
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation