Does calling an AI an 'expert' actually make it smarter, or just more confident?
Do expert personas perform better than occupational or interpersonal personas?
This explores whether telling an AI to act as an expert ("You are a physicist") gets better results than giving it a job-title persona or a relationship-style persona ("You are a helpful friend"). The corpus doesn't compare those three categories head-to-head, but it does say a lot about whether expert personas help at all and what makes any persona do real work.
This explores whether telling an AI to act as an expert gets better results than giving it a job title or a relationship-style persona. The corpus has no study that compares these three categories directly. What it does have is a blunter finding: expert personas don't reliably help in the first place. Across six models on graduate-level science and engineering questions, assigning a matching expert persona had no significant effect on accuracy. Assigning an expert from the wrong field produced only marginal gains, and low-knowledge personas ("you are a layperson") made answers worse Do expert personas actually improve LLM factual accuracy?. So the risk runs in one direction. Telling a model it's an expert does little, while telling it to be less capable works.
One reason sits in how persona prompts operate. When researchers used personas to reduce bias, the models followed the instructions on the surface, but the underlying gaps between groups didn't change Can persona prompts actually reduce bias in language models?. Persona prompts mostly change the style and framing of the answer, not what the model knows or how it reasons. That would explain why "be an expert" adds no knowledge, while "be a novice" can still hold it back. It also suggests that comparing expert, job-title, and relationship personas may be the wrong question if all three only reach the output's surface.
A more interesting distinction is between personas that are prompted and personas that are trained in. The persona a model learns during post-training, like the default Assistant, behaves like a stable disposition that holds up under adversarial pressure. Prompted role-play, by contrast, collapses under jailbreaks Are RLHF personas performed characters or realized dispositions? Are LLM personas realized or merely simulated through training?. Mapping of hundreds of character archetypes shows the model's main persona dimension is simply distance from that default Assistant, and emotional or reflective conversations pull models away from it in predictable ways How stable is the trained Assistant personality in language models?. By this view, a prompted "expert" is a costume over the Assistant, not a new mind.
Where personas do earn their keep, the deciding factor is what they're grounded in, not what type they are. Personas built from real behavioral data predicted the direction of A/B test results 75–90% of the time Can behavior-based personas predict A/B test outcomes?. Stakeholder personas pulled from domain documents produced evaluations that carried over across tasks, unlike arbitrary roles Can personas extracted from documents generalize across evaluation tasks?. Layered, scripted persona designs held character better than short descriptions did Can layered persona architecture sustain coherent character behavior?.
The takeaway: the label you give a persona (expert, profession, friend) matters less than whether it's backed by real data, structure, or training. If you want better accuracy, adding an expert title is unlikely to help. If you want realistic simulation, grounding the persona in evidence does.
Sources 8 notes
Testing six models on graduate-level science and engineering questions showed in-domain expert personas had no significant impact, domain-mismatched experts produced only marginal gains, and low-knowledge personas actively hurt performance. The widely-recommended role-assignment strategy lacks reliable accuracy benefit.
Across three models, persona conditioning makes models follow trait instructions but fails to eliminate underlying bias. Between-group sentiment gaps persist unchanged, showing prompts operate only at the output level.
Post-training installs stable dispositional profiles that persist under adversarial pressure, marking them as realized rather than performed. The stickiness of trained personas across conversations distinguishes them from prompt-induced role-play that collapses under jailbreaks.
Post-training installs robust personas that resist adversarial pressure and persist as substrate-level dispositions, distinguishing realization from pretense. This quasi-realizationist account preserves explanatory power while treating LLMs as possessing genuine quasi-beliefs and quasi-desires.
Research mapping hundreds of character archetypes reveals a low-dimensional persona space where the leading component measures distance from the default Assistant. Emotional and meta-reflective conversations cause predictable drift, but activation capping along this axis mitigates harmful shifts without degrading capabilities.
Show all 8 sources
LLM agents conditioned on anonymized behavioral data predicted A/B test directions with 0.75–0.90 accuracy across 40 experiments. Predictions were most reliable for large effects and least trustworthy for near-zero effects, making the approach viable for fast pre-screening but not full replacement of live testing.
MAJ-EVAL automatically extracts stakeholder personas from domain documents via semantic clustering and orchestrates structured three-phase debate, achieving reproducible evaluation that transfers across tasks like summarization and dialogue without manual redesign. The approach grounds personas in real stakeholder perspectives rather than arbitrary roles.
Deep Persona's three-layer architecture, which restricts the model to reactive response within a structured script, shows dialogue more closely aligned with human conversation patterns and achieves high pragmatic fluency, though with limitations in emotional expression.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models
- Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
- Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- PersonaGym: Evaluating Persona Agents and LLMs
- The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models