Why does role-play prompting work where explicit step-by-step fails?

The paper compares its role-play prompting method against Zero-Shot-CoT, the technique of appending "Let's think step by step" to a question. It splits its twelve benchmarks by whether ChatGPT already produces chain-of-thought (CoT) spontaneously under plain zero-shot prompting. "In tasks where the model struggles to generate CoT spontaneously, such as Last Letter, both our approach and Zero-Shot-CoT can stimulate CoT from scratch." But "for tasks where CoT already occurs, such as arithmetic, both our approach and Zero-Shot-CoT reinforce the step-by-step reasoning process, but Zero-Shot-CoT demonstrates no significant effect, whereas our approach leads to better performance." From this split the authors conclude: "we posit that role-play prompting is an implicit CoT trigger and can generate a more effective CoT in some fields compared with Zero-Shot-CoT."

The reasoning the paper gives rests on how ChatGPT differs from the earlier instruction-tuned models Zero-Shot-CoT was designed for: "different from the earlier instructed LLMs (Ouyang et al., 2022), the current conversational LLMs have undergone extensive supervised finetuning, which enables them to spontaneously generate CoT in some fields under the zero-shot setting." Where that spontaneous CoT already fires, telling the model to "think step by step" is redundant — the behavior is already present, so the explicit instruction adds nothing measurable. Role-play prompting's two-round dialogue (elaborating a role, then answering within it) changes more of the input context than a single appended phrase does, and the paper's claim is that this fuller framing reinforces the reasoning process the model would have produced anyway, rather than merely re-requesting it.

This complicates the same move — adding a fixed phrase or role to trigger reasoning — that Why do some questions perform better without step-by-step reasoning? examines from a different angle: that paper finds CoT triggers help or hurt depending on per-instance information flow between question and prompt, with some instances doing better without step-by-step reasoning at all. Both papers reject a uniform model of CoT triggers as context-free performance boosters; this paper's cut is by task-level spontaneity of CoT, the other's is by instance-level saliency dynamics. Set against Why do reasoning models lose character consistency during role-playing?, the "reinforcement" story told here is also scoped to a single short dialogue turn, not to the long internal reasoning traces where that later paper finds persona immersion actively competing with reasoning quality.

What the excerpt does not establish is a causal mechanism for the reinforcement effect — "we posit" signals the authors' own interpretation of a correlational pattern across benchmark categories, not a controlled test that isolates the persona's content from the two-round dialogue format, or from the fact that the role-feedback prompt is independently sampled and selected for quality. Nor does the excerpt report whether the same split (no effect of explicit CoT trigger where CoT is already spontaneous, effect from role-play) replicates on Vicuna or Llama 2-Chat, where only an aggregate "comparable gains" is mentioned. The finding should be read as evidence that CoT-triggering effects are task-dependent on at least one model family, not as a general account of when persona framing versus explicit instruction will win.