Do expert personas actually improve LLM task performance?

The paper distills three "desiderata" for persona prompting — Expertise Advantage, Robustness, and Fidelity — and benchmarks nine state-of-the-art open-weight LLMs across three model families and 27 tasks covering factual question answering, reasoning, and mathematics. The authors report that "expert personas usually lead to positive or non-significant performance changes," with success rates for the Expertise Advantage desideratum (the share of tasks where an expert persona does not underperform a no-persona baseline) ranging from 78% to 100% across models. But the same models are "highly sensitive to irrelevant persona details, with performance drops of almost 30 percentage points" — attributes such as a persona's name or favorite color, which by design should have no bearing on task performance, instead degrade it substantially in many cases.

The authors formalize Robustness as the worst-case utility across a set of task-irrelevant personas, borrowing from worst-group-accuracy evaluation in the robustness literature; if the desideratum holds, the metric should read zero. In practice it does not: irrelevant personas have a significant effect on performance "ranging from 14% (Qwen2-5.3B, color Robustness) to 59% (Llama 3.1-70B, color, and Llama3.1-8B, name Robustness) of the tasks," and in a smaller share of cases (3% to 14%, depending on the model) the effect runs the other way, with the no-persona baseline performing worse than all irrelevant personas tested. The authors state this robustness failure "is not solved by scaling up" on its own.

Do personas in prompts actually improve LLM factual accuracy? tested system-prompt personas on factual questions alone and found no average accuracy gain; this paper's broader task set (reasoning and math alongside QA) finds a similar near-zero-or-positive pattern for task-aligned expertise, but isolates a separate and sharper failure mode in task-irrelevant attributes that the factual-questions study did not centrally measure. Do expert personas actually improve LLM factual accuracy? reported that domain-matched experts had no significant effect; this paper's Fidelity results echo that inconsistency, finding domain-match and specialization-level effects "not significant" in most tasks even where the direction is positive more often than negative. Can persona prompts actually reduce bias in language models? found that persona conditioning holds or amplifies bias despite following surface-level trait instructions; the near-30-point swings from nominally irrelevant attributes reported here are consistent with that structural-limit framing, though this paper does not itself draw that connection.

The benchmark is restricted to tasks "with objectively verifiable ground truth" — factual QA, reasoning, and math — so the findings say nothing about persona effects in open-ended settings like creative writing or research ideation, which the authors flag explicitly as a limitation. Each tested persona also carries only a single attribute (expertise, name, or education level in isolation), not the combined, multi-attribute personas common in real deployments, so the reported robustness failures should be read as a floor on what multi-attribute personas could produce in practice, not an upper bound.