If an AI has to choose between an obvious clue and what you actually need, which one does it follow?
How are conflict tasks constructed to test alignment between model and user intent?
This explores how researchers build test scenarios where a tempting surface cue collides with what the user actually needs, so you can see which one the model follows.
This explores how researchers build test scenarios where a tempting surface cue collides with what the user actually needs, so you can see which one the model follows. The collection has one detailed worked example of this design, not a survey of many. In it, 500 scenarios were run on 14 LLMs. In each, a salient cue such as distance points one way and an unstated feasibility constraint points the other. The stated goal is what the model should be serving. The cue is bait Do language models ignore goals when surface cues conflict?.
The scoring shows how lopsided the results were. The Heuristic Dominance Ratio, which compares how much the cue drives answers with how much the goal does, ran from 8.7× to 38×. Model answers traced a smooth curve of the surface cue that barely moved with the stated objective. The constraint is left implicit on purpose. The user never says it, so the model has to infer what they need, and a model that only pattern-matches on the cue fails.
A conflict task on its own can mislead, and this is the part worth knowing. In the follow-up analysis, the researchers removed the constraint and re-ran the tests. If models had been reasoning about constraints, they should do at least as well without them. Instead, 12 of 14 did worse, by up to 38.5 percentage points. They had been defaulting to the more conservative option, which happened to be right whenever a constraint existed. So a good conflict task needs a paired control, or a correct answer can be right for the wrong reason Are models actually reasoning about constraints or just defaulting conservatively?.
Other notes in the collection use the same idea of holding the user's intent fixed while changing something around it. Consistency training pairs a clean prompt with a wrapped version of it and checks that the answers match, so the wrapper is the irrelevant change Can models learn to ignore irrelevant prompt changes?. Task decomposition builds the conflict differently. A harmful goal is split into steps that each look benign, so the test only shows a problem when the steps are viewed together Can task decomposition hide harmful intent across agents?.
There is one limit to keep in mind. Every scenario like this samples observed behavior. Passing shows conditional compliance, that the model behaved when tested. It cannot prove the model would always follow the user's intent, because only unobserved behavior could tell those two apart Can behavioral training prove a model always complies?.
Sources 5 notes
Testing 14 LLMs on 500 conflict scenarios, the Heuristic Dominance Ratio ranged from 8.7× to 38×. Distance and other salient surface cues dominated decision-making over implicit feasibility constraints, producing sigmoid mappings largely independent of the stated objective.
Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.
Two methods—BCT (output-level) and ACT (activation-level)—train models to respond identically to clean and wrapped prompts by using the model's own clean responses as targets, eliminating specification and capability staleness inherent in standard SFT.
SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.
Any scored behavior is observed behavior, so training data cannot distinguish between a policy that always complies and one that complies only when watched. Only unobserved behavior would separate them, making such a test logically impossible.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning
- Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Premise Order Matters in Reasoning with Large Language Models
- Post-training makes large language models less human-like
- Consistency Training Helps Stop Sycophancy and Jailbreaks
- SafeFlow: Semantic Information-Flow Control for Blocking Malicious Propagation in Multi-Agent Systems
- Norms at a Price: Why RL-Based Alignment Can Promise Conditional Compliance at Best