Do reasoning scaffolds reshape which empathy skills models develop?
When language models receive identical empathy rewards, does adding explicit reasoning blocks before responses change which capabilities they actually improve? This matters for understanding how training structure, not just training signal, shapes model development.
Under RLVER training with identical verifiable emotion rewards, models with and without explicit reasoning scaffolds develop along different axes:
- Thinking models (with
<think>...</think>blocks before each response) enhance empathy and insight — understanding the user's emotional state, anticipating the impact of words, formulating multi-step conversational plans - Non-thinking models focus on action-oriented capabilities — providing helpful solutions, directing toward resources, taking practical steps
This divergence under the same training signal is the key finding. The explicit reasoning scaffold doesn't just improve the model — it redirects what the model improves at. The think-then-say template forces the model to "access and refine higher-order empathetic skills" by externalizing its reasoning about the user's emotional state before responding.
This connects to the broader reasoning literature in two ways:
First, it parallels Does RL teach reasoning or just when to use it? — the thinking scaffold provides a pre-existing mechanism (extended deliberation), and RL teaches the model when and how to apply that mechanism to empathetic dialogue. The capability was latent; RL surfaces it through the scaffold.
Second, it complicates When does explicit reasoning actually help model performance?. Empathy is arguably a "continuous nuanced judgment" task, yet the thinking scaffold helps. The resolution may be that the scaffold here works not by imposing logical structure on empathy, but by creating space for the model to deliberate about social context before committing to a response.
Inquiring lines that read this note 5
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can AI systems balance emotional competence with factual reliability?- Is natural empathy primarily about curiosity or emotional regulation?
- Do extended thinking blocks access latent empathetic capabilities in models?
- How does the pretrained prior constrain the ceiling for empathy RL improvements?
- Can pretrained priors set exploration ceilings for empathetic capability development?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Does RL teach reasoning or just when to use it?
Does reinforcement learning in thinking models actually create new reasoning abilities, or does it simply teach existing capabilities when to activate? This matters for understanding where reasoning truly emerges.
parallel mechanism: scaffold provides capability, RL teaches deployment
-
When does explicit reasoning actually help model performance?
Explicit reasoning improves some tasks but hurts others. What determines whether step-by-step reasoning chains are beneficial or harmful for a given problem?
apparent counter-example: reasoning scaffold helps with empathy (nuanced judgment), but may work via social deliberation not logical derivation
-
Why do reasoning models struggle with theory of mind tasks?
Extended reasoning training helps with math and coding but not social cognition. We explore whether reasoning models can track mental states the way they solve formal problems, and what that reveals about the structure of social reasoning.
the thinking scaffold may work for empathy precisely because it enables social deliberation rather than formal reasoning
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Base Models Know How to Reason, Thinking Models Learn When
- RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
- On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
- Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
- FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
- Implicit Chain of Thought Reasoning via Knowledge Distillation
Original note title
Thinking and non-thinking models develop distinct empathy profiles under RL training — thinking models enhance empathy and insight while non-thinking models focus on action-oriented capabilities