Role-play prompting boosts AI accuracy a lot — but is it the character, or just the extra conversation around it?

This explores whether role-play prompting's accuracy boost comes from the character itself or from the extra conversation built around it. The corpus leans toward the conversation. The headline result is big: framing reasoning questions inside a two-stage role-play dialogue raised ChatGPT's zero-shot accuracy by 10 to 60 percentage points across twelve benchmarks Does role-play dialogue boost reasoning accuracy in ChatGPT?. But when other researchers isolated the persona, simply adding "You are a lawyer" or any of 162 other identities to a system prompt, the average accuracy gain on factual questions was zero, and no method could predict which persona would help on which question Do personas in prompts actually improve LLM factual accuracy?. If the identity alone did the work, those two findings wouldn't fit together.

The likelier mechanism is the dialogue structure. In the role-play studies, adding "think step by step" did nothing in cases where the model already reasoned that way on its own, yet role-play still improved accuracy Why does role-play prompting work where explicit step-by-step fails?. A plausible reading is that the two-round exchange, where the model first accepts a role and then gets the question, strengthens reasoning the model already does instead of adding a new instruction. This matches a broader limit: prompting can only bring out knowledge and skills the model already has. It can't teach new ones Can prompt optimization teach models knowledge they lack?. So role-play gains are best read as better access to existing ability, not new capability.

A surprising detail: what seems to matter is who the model thinks it's talking to more than who it's told to be. Telling the model about its audience ("You are talking to a lawyer") slightly beat assigning a speaker identity ("You are a lawyer") across nine models Do audience-framed prompts work better than speaker-framed prompts?. The effect is small, but it fits the idea that a prompt is doing more than giving an instruction. It stands in for the shared context that two people would normally build up over a conversation, all set at once in a frame the model can't renegotiate How do prompts reshape the role of context in AI conversation?. Role-play dialogue may work because it builds that context gradually, the way human conversation does.

The same confusion between "role" and "context" shows up in how role-play agents are evaluated. Benchmarks that score agents on continuations of a fixed dialogue history mix up the agent's own skill with the context it's handed. When each agent is paired with simulated users built from real conversation histories instead, no single agent wins for every user Does fixed dialogue history bias role-play agent evaluation?. So when anyone reports that a role "works", it's fair to ask how much of the credit belongs to the surrounding conversation.

One more angle on why personas behave this way: a model doesn't commit to a single character. It samples from a range of characters that fit the context so far Do large language models actually commit to a single character?. A persona prompt narrows that range, but only at the surface. Persona conditioning changes the style of the output while the underlying bias stays the same Can persona prompts actually reduce bias in language models?. Put together, the gains seem to come from how the role is set up, not from the role itself. The corpus has no head-to-head study that adds dialogue turns without a role, so that direct comparison remains open.