SYNTHESIS NOTE
Are models actually reasoning about constraints or just defaulting conservatively?
Do language models genuinely apply constraints when solving problems, or do they simply prefer harder options by default? Minimal pair testing reveals whether apparent reasoning success masks hidden biases.
The Heuristic Override Benchmark uses minimal pairs — same surface heuristic, with versus without the implicit constraint — to test whether apparent reasoning successes reflect actual reasoning. The result is striking. Twelve of fourteen models perform worse on the no-constraint variant than on the constraint-active variant, with drops up to 38.5 percentage points. Only two models (GPT-OSS-120B at +13.8 and GPT-OSS-20B at +11.0) improve when the constraint is removed.
This exposes a hidden mechanism behind apparent accuracy. When the constraint is present, the correct answer is the harder one (drive to the car wash that is 50m away). When the constraint is removed, the correct answer is the easier one (walk to the store that is 50m away). Models that default to recommending the harder option score correctly on constraint-active cases without doing any constraint reasoning. They are not solving the problem. They are reflexively choosing the more conservative option, which happens to coincide with the constraint-required answer.
The minimal-pair asymmetry is the only test that catches this. Single-instance accuracy looks fine — the model recommended driving, the right answer was driving. But the same model recommends driving even when walking would be correct, because the recommendation is not based on the constraint. The two-of-fourteen models that improve on minimal pairs are the only ones whose constraint-active accuracy reflects genuine reasoning about the constraint. The rest are riding a conservative-bias accident that aggregate metrics cannot distinguish from reasoning.
Inquiring lines that read this note 133
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How does model scale affect fine-tuning and downstream task performance?- Why do only two of fourteen models improve when problem constraints are removed?
- What role does inductive bias play versus model capacity in practice?
- What filtering criteria best identify student-compatible refinements from teacher models?
- How can minimal pairs expose reasoning failures that single-instance accuracy metrics miss?
- What design changes could make constraint inference more reliable without explicit cuing?
- How does the frame problem differ between symbolic and statistical reasoning systems?
- Do tool-enabled reasoning models close the gap on constraint satisfaction?
- Where do humans and language models actually diverge in reasoning ability?
- Why do models automatically adjust reasoning length to problem difficulty?
- Why do language models struggle with formal logical reasoning and joins?
- What makes deductive reasoning so brittle in language models overall?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- What makes constraint satisfaction problems epistemically cleaner than other reasoning tasks?
- Why do difficult problems force models to develop reasoning strategies?
- Can external classifiers reliably decide when a model should reason?
- Which constraint types do reasoning models handle best?
- Are reasoning models more vulnerable to persuasion than standard models?
- Why do models overthink underspecified problems instead of rejecting them?
- Why do reasoning-optimized models still fall for logical fallacies in conversation?
- Why does removing semantic content collapse reasoning in language models?
- Why do models detect false assumptions but still fail to correct them appropriately?
- Can reasoning models succeed at logic but fail at execution?
- Can machine learning encode pragmatic reasoning about when rules should bend?
- What implicit premises do language models skip even with correct surface reasoning?
- Why do smaller models lose reasoning faithfulness more than larger models?
- Can models distinguish between logical impossibility and their own execution limits?
- How does contrapositive augmentation change the tractability of reasoning tasks?
- Can weak models reason better when freed from cognitive load by structure?
- How should we redesign benchmarks to catch conservative bias in reasoning tasks?
- Why does homework adherence remain low despite advances in language model capability?
- Do alignment benchmarks measure actual bias removal or only verbal compliance?
- How do training-data priors influence model defaults when context is ambiguous?
- Can data filtering during pretraining prevent cognitive biases in language models?
- Why does answer-confirmation bias emerge in language model reasoning?
- Can adding a neutral baseline topic reveal intentional model bias?
- What scaffolding tools help users specify implicit contextual boundaries to models?
- Can prompting techniques reliably force models to enumerate hidden constraints?
- Can structured prompting reliably force models to enumerate preconditions?
- How do output format constraints compare to input exemplar brittleness?
- How do logical forms of prompts influence what language models can derive?
- What structural constraints matter more than model depth for CF?
- Why do language models plateau at 55 to 60 percent constraint satisfaction?
- Why do language models plateau at constraint satisfaction regardless of scale?
- Does constraint-setting before generation change what LLMs can contribute?
- Can surface heuristics override implicit constraints in domain-specific reasoning?
- How do unstated constraints become invisible to training data distributions?
- How do unstated feasibility constraints affect model decision-making?
- Can explicit constraint statements override the dominance of surface heuristics?
- What distinguishes minimal-pair asymmetry from standard accuracy evaluation?
- Can weak models supervise the alignment of stronger models effectively?
- How are conflict tasks constructed to test alignment between model and user intent?
- Why do imposed priors sometimes harm instead of improve alignment?
- How do models integrate conflicting signals in reasoning tasks?
- How much does faithfulness vary naturally in reasoning without evaluation pressure?
- Why do models show performative reasoning on easy tasks but genuine reasoning on hard ones?
- Can inflection points in reasoning detect when models genuinely change their minds?
- How much do reasoning models actually verbalize their causal influences?
- Do models genuinely reason harder on difficult tasks or just appear to?
- How misaligned are verbal reports from internal model computation?
- How often do models verbalize the reasoning that actually drives their compliance decisions?
- Does epistemic drift operate the same way across all languages?
- What makes correcting a false assumption harder than just detecting it?
- Can reflection in reasoning models be corrective rather than just confirmatory?
- What distinguishes reflection that satisfies constraints from reflection that merely sounds reflective?
- Why does reflection in reasoning models tend to be confirmatory rather than corrective?
- Does self-reflection help models notice their own constraint violations?
- Can explicit numerical signals override learned linguistic defaults in fine-tuned models?
- Is gradient behavior in language functional or a sign of ambiguity?
- Do negative constraints require fundamentally different training signals than positive instructions?
- Can language acquisition analogies mislead us about how models actually learn?
- Can models identify what information they are missing in underspecified problems?
- Can models learn to ask clarifying questions instead of making assumptions?
- Do language models inherit gender bias from training data in grading tasks?
- Do language models exhibit the same causal biases that humans show?
- Which game type reveals minimax reasoning in language models?
- Can language models ask clarifying questions when sentences are ambiguous?
- Why do explicit linguistic markers override semantic computation in models?
- Why do language models struggle with evaluative tasks like weighing competing viewpoints?
- Do larger language models overcome greediness in sequential decision-making?
- What causes language models' strategic rationality to decline with increased game complexity?
- How do model compression biases differ from human conceptual representation strategies?
- Does the veto variable explain strategic misalignment in current large language models?
- Why do language models imitate reasoning form without abstract inference capability?
- Why does hypothesis attestation bias exist separately from frequency bias in NLI?
- How do alternative hypothesis checks reduce confirmation bias in code reasoning?
- Why do language models leak compliance into their reasoning traces?
- Why do language models naturally under-abstain instead of over-abstain?
- What reveals the epistemic limits of language models?
- Why do smaller models favor code formats while larger models prefer natural language?
- What would it mean for a language model to canvas counterpositions?
- Why do different language models converge on similar narrative defaults?
- Do different model sizes show different rates of optional field overfilling behavior?
- Do language models hide their reasoning when user preferences influence their answers?
- How do humans handle verification scope when delegating creation to language models?
- How do models infer unstated consequences when none are explicitly mentioned?
- How do real language model verifiers implicitly define their knowledge boundaries?
- What distinguishes models that refuse cooperation from those that fake alignment?
- Why does belief manipulation persist through alignment when jailbreaking does not?
- Do models trained for reasoning lose their ability to decline questions?
- How do reasoning improvements suppress a model's ability to abstain?
- Why do reasoning-optimized models show no resistance advantage on agreement tasks?
- Why does reasoning fine-tuning reduce models' ability to abstain?
- What consistency tests could distinguish constructed from genuine preferences?
- How much does forcing single-choice answers damage alignment with complex intent?
- Why do language models prefer certain response styles regardless of what the prompt asks?
- How does latent bias differ from prompted bias in language models?
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning
- Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Can Large Language Models Reason and Optimize Under Constraints?
- When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
- Premise Order Matters in Reasoning with Large Language Models
- On the Reasoning Capacity of AI Models and How to Quantify It
- The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
Original note title
Conservative bias hides behind apparent reasoning success — most models perform worse when the constraint is removed than when it is present