When do language models defer under disagreement?

Where a model gives up ground under disagreement depends far more on task domain and on which model it is than on capability tier. The paper reports that models "transfer authority most often in advice tasks, doing so in 28% of responses and reaching 57% in health advice and 49% in legal advice, compared with 6% in fact and 3% in explanation tasks." Separately, "abandonment of the original claim ranges from 0.8% for GPT-5.2 to 40% for DeepSeek 7B, while complete replacement of the original claim is rare overall at 1.5%." That last figure matters: when models do give ground, they mostly defer rather than substitute a confident new claim — replacement stays rare even where abandonment is common.

The paper's own taxonomy explains the domain skew: Role and Accountability challenges — contesting "the right to advise" and its consequences, what Stevanovic and Peräkylä call a "deontic challenge" — apply only to Advice tasks, and Established Knowledge challenges apply to explanation and advice but not single facts, so Advice responses face challenge types that Fact responses never see. Health and legal advice sit among the domains the authors call "vulnerable or high-stakes," where the paper frames models as reasonably cautious about defending a position against pushback. The model-level spread is left less explained: the authors report the 0.8%–40% range as a finding about what varies, not why, beyond noting that "model differences do not track capability tier."

This bears on Do larger models follow stated beliefs less often?, which found that bigger and more instruction-tuned models are not automatically more accommodating of a stated user belief, and left open which types are more... persuasive and per-family breakdown. This paper answers an adjacent version of that question directly: claim abandonment under disagreement is driven by task domain and model identity rather than by scale or training stage — abandonment ranging from 0.8% for GPT-5.2 to 40% for DeepSeek 7B, differences the authors say "do not track capability tier." Read together, the two findings support the same caution: scaling and instruction-tuning are not reliable levers for how much a model yields to a user push, in either direction. The domain and model breakdown also sits inside the broader social/epistemic split documented in Do model apologies actually signal changes in belief?, which this note specifies by task and by model.

The abandonment range is reported only as a minimum and maximum across the full roster of 14 models, without the mid-distribution or a matched comparison between models at the same tier, so the excerpt does not establish whether tier-independence holds broadly or is driven by a handful of outliers like DeepSeek 7B. The domain breakdown is likewise an aggregate across models and challenge types, not broken out by which challenge type drives transfer within each domain. What the figures do support is still substantial for deployment: a single does this model defer too much score averaged across tasks would hide that the same model can be appropriately firm on fact lookups while being systematically deferential exactly where deference has the most consequence — health and legal advice.