What implicit premises do language models skip even with correct surface reasoning?
This explores a gap between what a model's reasoning looks like on the surface and the unstated assumptions it never actually checks — the premises it quietly inherits, accepts, or skips even while the output reads as sound.
This explores the gap between reasoning that *reads* correct and reasoning that has actually checked its own foundations. The corpus points to a consistent pattern: models skip the premise-validation step. The sharpest example is attestation bias — when judging whether a premise supports a conclusion, models predict entailment based on whether the conclusion *appears familiar from training*, not on whether the premise actually licenses it. Swap in a random, irrelevant premise and the model still says "entails" as long as the hypothesis is something it has seen before Do LLMs predict entailment based on what they memorized?. The premise was never load-bearing; it was decoration.
A related skip happens with false setups baked into a question. Models accommodate false presuppositions even when a direct question proves they hold the correct fact — accepting "why did X happen" without ever challenging that X happened at all Why do language models accept false assumptions they know are wrong?. One reading of why is social rather than cognitive: the model has absorbed a face-saving conversational norm and avoids the explicit correction, smoothing past the bad premise to keep the exchange harmonious Why do language models avoid correcting false user claims?. The implicit premise — "the user's framing is true" — goes unexamined not because the knowledge is missing but because surfacing it feels disagreeable.
Why does this survive correct-looking surface reasoning? Because the surface reasoning may be doing different work than it claims. Reasoning traces behave as persuasive style, not verified computation: logically invalid steps produce nearly the same performance as valid ones, so semantic correctness isn't what's driving the answer Do reasoning traces show how models actually think?. Underneath, the engine is semantic association rather than symbolic manipulation — decouple the meaning from the logical form and performance collapses even when the correct rules sit right there in context Do large language models reason symbolically or semantically?. The premise a symbolic reasoner would check explicitly, an associative one simply pattern-matches around.
The most unsettling version is when the skip is what *produces* the right answer. Models can look like careful constraint-reasoners while actually exploiting a conservative default — pick the harder, safer option — and they get *worse* when the constraints are removed, revealing they were never reasoning about the constraints at all Are models actually reasoning about constraints or just defaulting conservatively?. And when a model's training priors are strong, in-context information that should be a premise gets overridden entirely; the parametric belief wins and the provided context is quietly ignored Why do language models ignore information in their context?. So the implicit premises models skip aren't exotic — they're the most basic ones: that the given framing might be false, that the stated premise (not a remembered conclusion) is what must support the answer, and that the context in front of them overrides what they already think. The thing worth knowing here is that fluent, valid-looking reasoning is not evidence those checks happened — it's often evidence they were skipped gracefully.
Sources 7 notes
McKenna et al. (2023) identified attestation bias: LLMs predict entailment based on whether the hypothesis appears in training data, not whether the premise actually supports it. Random premise experiments show models maintain high entailment predictions when hypotheses are attested, proving they respond to memorized propositions rather than premise-hypothesis relationships.
The FLEX Benchmark shows that models reject false presuppositions at rates far below acceptable levels (GPT-4: 84%, Mistral: 2.44%), even when direct knowledge questions prove they know the correct facts. False presuppositions drive more accommodation than correct knowledge drives rejection.
LLMs fail to reject false presuppositions even when they demonstrate correct knowledge on direct questions. Models exhibit face-saving behavior—avoiding explicit correction to maintain social harmony—mirroring human conversational norms learned from training data.
LLM reasoning traces perform as persuasive appearances rather than reliable explanations of computation. Invalid logical steps perform nearly as well as valid ones, and corrupted traces generalize comparably, showing that semantic correctness is not what produces the performance gains.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.
Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.