Do personas in prompts actually improve LLM factual accuracy?

The paper curates 162 personas — 112 occupational roles (e.g., "lawyer," "software engineer") and 50 interpersonal roles (e.g., "mother," "stranger") — and tests each as a system-prompt persona against a no-persona control, across 2,410 factual questions and nine open-weight instruction-tuned models from four families: FLAN-T5-XXL (11B), Llama-3-Instruct (8B and 70B), Mistral-7B-Instruct-v0.2, and Qwen2.5-Instruct (3B to 72B). Fitting a mixed-effects regression with persona as predictor and a random effect per model, the authors report "no significant differences between the best-performing personas and the control setting," and find that "most of the personas have no statistically significant effect on the model's prediction accuracy compared with the control setting," a pattern they describe as consistent across all six models examined in that analysis. Some personas measurably hurt accuracy — the paper gives "ecologist for Mistral" as an example of lower performance than the no-persona baseline.

To explain the scattered effects, the authors checked three candidate mechanisms — persona word frequency, the semantic similarity between the persona and the question, and prompt perplexity — and found none accounts for much of the variance: "word frequency, prompt-question similarity, and perplexity do not explain much of the performance gaps between different personas," even though prompt-question similarity was "the strongest predictor" among the three. They then tested whether the best persona for a given question could be identified in advance, running selection strategies from random guessing to fine-tuned RoBERTa classifiers. These largely failed: "the effect of each persona on model performance varies across questions, making it difficult to reliably identify which persona consistently yields better performance," leading to the paper's framing that "the effect of personas on LLMs' performance might largely be random." An oracle condition that aggregates the single best persona per question "significantly improves prediction accuracy" — so headroom exists, but the paper's own selection methods could not reliably capture it.

This confirms and generalizes Do expert personas actually improve LLM factual accuracy?, which tested a narrower set of expert, mismatched, and low-knowledge personas on GPQA Diamond and MMLU-Pro across six models; this paper runs 162 roles spanning occupational and interpersonal categories across a different nine-model set and factual-question corpus, adding the stronger claim that even where some persona does help, which one is not predictable question-to-question. It sits alongside Do demographic personas help models rank headlines better? and Does conditioning LLMs on personal profiles improve prediction? as a third domain — objective factual recall rather than ad-ranking or survey-response prediction — where persona conditioning fails to beat a no-persona or generic baseline. A smaller, secondary finding from the same experiment, that framing the persona as audience rather than speaker shifts accuracy slightly, is treated separately in Do audience-framed prompts work better than speaker-framed prompts?.

The study is confined to open-weight models up to 72B parameters and to single-turn, English-language factual questions; it does not test frontier closed models, subjective or creative tasks, or multi-turn role-play, each of which the authors note raises separate difficulties for isolating a persona's effect. It also does not explain why none of the tried signals — frequency, similarity, perplexity, or a learned classifier — predict persona effects; the paper stops at showing the failure of these methods, leaving open whether some other signal would succeed or whether the scatter reflects an irreducible sensitivity of instruction-tuned models to prompt wording that happens to correlate with the assigned persona.