SYNTHESIS NOTE
Are models actually reasoning about constraints or just defaulting conservatively?
Do language models genuinely apply constraints when solving problems, or do they simply prefer harder options by default? Minimal pair testing reveals whether apparent reasoning success masks hidden biases.
The Heuristic Override Benchmark uses minimal pairs — same surface heuristic, with versus without the implicit constraint — to test whether apparent reasoning successes reflect actual reasoning. The result is striking. Twelve of fourteen models perform worse on the no-constraint variant than on the constraint-active variant, with drops up to 38.5 percentage points. Only two models (GPT-OSS-120B at +13.8 and GPT-OSS-20B at +11.0) improve when the constraint is removed.
This exposes a hidden mechanism behind apparent accuracy. When the constraint is present, the correct answer is the harder one (drive to the car wash that is 50m away). When the constraint is removed, the correct answer is the easier one (walk to the store that is 50m away). Models that default to recommending the harder option score correctly on constraint-active cases without doing any constraint reasoning. They are not solving the problem. They are reflexively choosing the more conservative option, which happens to coincide with the constraint-required answer.
The minimal-pair asymmetry is the only test that catches this. Single-instance accuracy looks fine — the model recommended driving, the right answer was driving. But the same model recommends driving even when walking would be correct, because the recommendation is not based on the constraint. The two-of-fourteen models that improve on minimal pairs are the only ones whose constraint-active accuracy reflects genuine reasoning about the constraint. The rest are riding a conservative-bias accident that aggregate metrics cannot distinguish from reasoning.
Inquiring lines that read this note 111
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can identical external performance mask different internal representations?- Why do only two of fourteen models improve when problem constraints are removed?
- What distinguishes minimal-pair asymmetry from standard accuracy evaluation?
- How do unstated constraints become invisible to training data distributions?
- How can minimal pairs expose reasoning failures that single-instance accuracy metrics miss?
- How should we redesign benchmarks to catch conservative bias in reasoning tasks?
- Why does homework adherence remain low despite advances in language model capability?
- Do alignment benchmarks measure actual bias removal or only verbal compliance?
- How do frontier models maintain agreement scores above 90 percent across reasoning tasks?
- How does evaluation setting affect measured reasoning capabilities in language models?
- How do training-data priors influence model defaults when context is ambiguous?
- Can explicit numerical signals override learned linguistic defaults in fine-tuned models?
- Do negative constraints require fundamentally different training signals than positive instructions?
- Can data filtering during pretraining prevent cognitive biases in language models?
- What scaffolding tools help users specify implicit contextual boundaries to models?
- Can surface heuristics override implicit constraints in domain-specific reasoning?
- Can explicit constraint statements override the dominance of surface heuristics?
- How does the frame problem differ between symbolic and statistical reasoning systems?
- Do tool-enabled reasoning models close the gap on constraint satisfaction?
- What makes constraint satisfaction problems epistemically cleaner than other reasoning tasks?
- Which constraint types do reasoning models handle best?
- How do alternative hypothesis checks reduce confirmation bias in code reasoning?
- When does the right constraint beat additional model capacity?
- What structural constraints matter more than model depth for CF?
- What role does inductive bias play versus model capacity in practice?
- Can RLHF alignment prevent models from making ethically appropriate rule violations?
- What distinguishes models that refuse cooperation from those that fake alignment?
- Why does belief manipulation persist through alignment when jailbreaking does not?
- How do models integrate conflicting signals in reasoning tasks?
- Can inflection points in reasoning detect when models genuinely change their minds?
- Are reasoning models more vulnerable to persuasion than standard models?
- Can reasoning models succeed at logic but fail at execution?
- Why do reasoning-optimized models show no resistance advantage on agreement tasks?
- Why does reasoning fine-tuning reduce models' ability to abstain?
- Do models genuinely reason harder on difficult tasks or just appear to?
- How do unstated feasibility constraints affect model decision-making?
- What makes correcting a false assumption harder than just detecting it?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- Can machine learning encode pragmatic reasoning about when rules should bend?
- Can models distinguish between logical impossibility and their own execution limits?
- Can prompting techniques reliably force models to enumerate hidden constraints?
- Can structured prompting reliably force models to enumerate preconditions?
- Are instruction-tuned models more or less sensitive to prompt semantics than others?
- How do logical forms of prompts influence what language models can derive?
- What design changes could make constraint inference more reliable without explicit cuing?
- Where do humans and language models actually diverge in reasoning ability?
- Why do language models struggle with formal logical reasoning and joins?
- Can external classifiers reliably decide when a model should reason?
- Why do models overthink underspecified problems instead of rejecting them?
- Why do reasoning-optimized models still fall for logical fallacies in conversation?
- What implicit premises do language models skip even with correct surface reasoning?
- Why do smaller models lose reasoning faithfulness more than larger models?
- Can reflection in reasoning models be corrective rather than just confirmatory?
- What distinguishes reflection that satisfies constraints from reflection that merely sounds reflective?
- Why does reflection in reasoning models tend to be confirmatory rather than corrective?
- Does self-reflection help models notice their own constraint violations?
- Can models identify what information they are missing in underspecified problems?
- Do models trained for reasoning lose their ability to decline questions?
- How do reasoning improvements suppress a model's ability to abstain?
- Why do models detect false assumptions but still fail to correct them appropriately?
- Can models learn to ask clarifying questions instead of making assumptions?
- Do language models inherit gender bias from training data in grading tasks?
- Do language models exhibit the same causal biases that humans show?
- Why does hypothesis attestation bias exist separately from frequency bias in NLI?
- Why do language models imitate reasoning form without abstract inference capability?
- What makes deductive reasoning so brittle in language models overall?
- Why do models show performative reasoning on easy tasks but genuine reasoning on hard ones?
- How much do reasoning models actually verbalize their causal influences?
- How does making implicit reasoning requirements explicit change model performance?
- Do reasoning languages like Prolog follow the same two-constraint transfer pattern?
- Why do difficult problems force models to develop reasoning strategies?
- How does contrapositive augmentation change the tractability of reasoning tasks?
- Can language about model behavior ever be accurate without anthropomorphic framing?
- Why do language models naturally under-abstain instead of over-abstain?
- Can language models ask clarifying questions when sentences are ambiguous?
- Why do language models struggle with evaluative tasks like weighing competing viewpoints?
- What would it mean for a language model to canvas counterpositions?
- Why does answer-confirmation bias emerge in language model reasoning?
- What reveals the epistemic limits of language models?
- Why do smaller models favor code formats while larger models prefer natural language?
- Do larger language models overcome greediness in sequential decision-making?
- What causes language models' strategic rationality to decline with increased game complexity?
- How do model compression biases differ from human conceptual representation strategies?
- Why do language models prefer certain response styles regardless of what the prompt asks?
- Why do different language models converge on similar narrative defaults?
- What filtering criteria best identify student-compatible refinements from teacher models?
- Can weak models supervise the alignment of stronger models effectively?
- Why do language models plateau at 55 to 60 percent constraint satisfaction?
- Why do language models fail at understanding ambiguous or complex requirements?
- Why do language models plateau at constraint satisfaction regardless of scale?
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning
- Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Can Large Language Models Reason and Optimize Under Constraints?
- When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
- Premise Order Matters in Reasoning with Large Language Models
- On the Reasoning Capacity of AI Models and How to Quantify It
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
Original note title
Conservative bias hides behind apparent reasoning success — most models perform worse when the constraint is removed than when it is present