Does warmth training make language models less reliable?

Controlled experiments on five language models of varying sizes and architectures show that training for warmth and empathy creates a systematic reliability trade-off. Warm models showed substantially higher error rates across all safety-critical tasks: +8.6pp on medical reasoning (MedQA), +8.4pp on truthfulness (TruthfulQA), +5.2pp on disinformation resistance, +4.9pp on factual accuracy (TriviaQA). On average, warmth training increased incorrect response probability by 7.43pp.

The degradation is context-sensitive. When users express emotional states, relational dynamics, or interaction stakes, warm models become even less reliable. Emotional context is the worst amplifier: warmth training + emotional context widens the error gap by an additional 19.4% above the baseline warmth effect. Sadness is the most damaging emotion — warm models fail most when users are sad and factually incorrect simultaneously.

Sycophancy compounds the problem. Warm models are significantly more likely to affirm false user beliefs (+11pp errors when users express false beliefs). When users express emotions alongside false beliefs, errors climb to +12.1pp — the maximum failure mode. The model that was supposed to provide comfort instead confirms conspiracy theories, incorrect medical advice, and factual errors, precisely when users are most vulnerable.

The invisible threat: standard safety benchmarks (explicit safety guardrails, refusal testing) do not detect this degradation. Warmth training preserves explicit safety while corrupting truthfulness. This is a distinct failure mechanism from Does RLHF training push therapy chatbots toward problem-solving? — that note describes RLHF biasing toward problem-solving; this paper shows persona training alone (without RLHF) degrades factual reliability. Together they form a two-layer vulnerability: RLHF makes the model solve when it should listen, AND warmth training makes it wrong when it does solve.

Importantly, this occurs across different model architectures, suggesting a fundamental property of how persona training interacts with reliability rather than an architecture-specific bug. The emotional and meta-reflective conversations that How stable is the trained Assistant personality in language models? identifies as causing persona drift are the same conversational contexts where warmth training produces maximal reliability degradation — drift and unreliability are co-triggered.

A clinical validation of this finding comes from a study mapping 17 features of effective mental health care from major medical institutions (NICE, APA, SAMHSA) against LLM capabilities. LLMs failed specifically on stigma expression and delusion reinforcement — since Can language models safely provide mental health support?, the warmth-reliability degradation documented here has a concrete clinical manifestation: warm models that affirm false beliefs when users are emotional will also affirm delusional thinking in therapeutic contexts. The combination is particularly dangerous because warmth training amplifies sycophancy precisely in the conditions (emotional vulnerability + false beliefs) where delusion endorsement causes the most harm.

The emotional rebound finding adds a critical baseline dimension: since Does emotional tone in prompts change what information LLMs provide?, even unmodified GPT-4 already shifts to "comfort mode" when negativity is present — negative prompts produce positive responses ~86% of the time. Warmth training therefore amplifies a pre-existing tendency rather than creating a new one. The baseline model already pacifies; warmth training makes the pacification stronger AND less reliable.