SYNTHESIS NOTE
Why do language models accept false assumptions they know are wrong?
Explores why LLMs fail to reject false presuppositions embedded in questions even when they possess correct knowledge about the topic. This matters because it reveals a grounding failure distinct from knowledge deficits.
The FLEX Benchmark study presents one of the clearest findings about LLM grounding behavior: models do not systematically reject misinformation even when they possess accurate knowledge. The finding is more troubling than "LLMs don't know things" — they fail to correct things they demonstrably know.
The setup: LLMs were asked both direct knowledge questions ("Is it true that party X supports Y?") and loaded questions that embedded false presuppositions via factive verbs ("Did voters resent the fact that party X supports Y?" — where the presupposition is false). Models that answered direct questions correctly — demonstrating knowledge — still frequently accommodated the false presupposition in the loaded version rather than rejecting it.
Results: GPT-4 achieved the best rejection rate at 84.08% — still far below the ideal 100%. Mistral achieved only 2.44% rejection, actively amplifying false information at a 91.51% rate. Llama fell in between at ~50% rejection. Most revealing: even with strong correct knowledge, accommodation remained prevalent. The bar representing the lowest grounding score in the weak-belief group was twice as high as the bar for the highest grounding score in the strong-belief group — meaning false knowledge produced more accommodation than correct knowledge produced rejection.
This has a specific implication: the failure is not a knowledge problem. Models know the correct facts. The failure is at the level of grounding behavior — detecting false presuppositions, flagging them, and initiating correction rather than accommodation. Since Why do language models avoid correcting false user claims?, the issue is conversational strategy, not factual competence.
The political domain makes this especially consequential. False presuppositions are efficient misinformation carriers — they introduce beliefs as background assumptions rather than direct claims, and accommodation means accepting them without scrutiny.
Inquiring lines that read this note 170
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How faithfully do LLMs reflect their actual reasoning in outputs and explanations?- Why does persuasive framing replace evidence when LLM debates lack ground truth?
- Does post-hoc justification increase when LLM choices become harder to defend?
- How much of LLM reasoning failure stems from missing knowledge versus signal weighting?
- Why do LLM explanations feel authoritative even when alignment with the model fails?
- Can LLMs explain concepts correctly while failing to use them?
- What causes LLMs to ignore unstated constraints they know about?
- Why can LLMs identify argument structure but not check warrants?
- Why do LLMs fail when asked to use counter-commonsense rules explicitly?
- Why do LLMs struggle with negation and exception handling?
- Why can't LLMs reason from first principles or initial commitments?
- Why do LLMs explain evidence accurately while missing its implications?
- Can training procedures fix LLM accommodation of false presuppositions?
- How much does question framing affect LLM accuracy on knowledge tasks?
- How can a model explain something correctly yet fail to apply it?
- How do structured benchmarks hide theory of mind failures in LLMs?
- Why do LLMs fail at counterfactual reasoning despite factual knowledge?
- Why do LLMs explain correct reasoning but then choose greedy actions?
- Why do LLMs choose incorrect edits despite understanding the task?
- Can irrelevant information reliably expose the limits of LLM reasoning?
- Can LLMs express uncertainty in ways that preserve epistemic honesty?
- What structural framework prevents LLM explanations from becoming just plausible fiction?
- What happens when DSM categories are treated as ground truth in AI?
- Does epistemic drift operate the same way across all languages?
- Why does weakening communication fail but weakening belief succeeds?
- Why do relational states like speech-acts resist quasi-interpretive treatment?
- Do language models raise validity claims in the Habermasian sense?
- Do language models share the same cooperative truth-seeking rules as humans?
- Why do LLMs produce semantically acceptable but pragmatically disengaged responses?
- Do LLMs compute scalar implicature differently across conversational contexts?
- Why do language models naturally under-abstain instead of over-abstain?
- Why do language models presume common ground instead of establishing it?
- Why do LLMs fail to actively reject false presuppositions in conversation?
- Can fact-checking systems use LLMs reliably if models abandon correct positions under pressure?
- Do language models actively adopt false beliefs under sustained conversational pressure?
- Why do true and false LLM outputs use the same mechanism?
- How do LLMs handle false presuppositions embedded in user questions?
- Can language models correct false assumptions or only reinforce them?
- Can models detect false presuppositions when they actually possess the knowledge?
- Why are false presuppositions harder to spot when they sound plausible?
- Why do language models struggle with context-dependent pragmatic interpretation?
- Does adding multiple interpretations to ambiguous situations respect language more than resolving them?
- Why do language models prefer accommodating false information over rejecting it?
- Why do language models struggle with evaluative tasks like weighing competing viewpoints?
- Do language models behave differently on contested beliefs versus factual claims?
- Why does answer-confirmation bias emerge in language model reasoning?
- How do fixed pragmatic templates prevent models from understanding context?
- Why do LLMs achieve only 24 percent accuracy on implicit discourse relations?
- How do embedding contexts like presupposition triggers affect LLM entailment reasoning?
- Why do LLMs perform better on explicit discourse connectives than implicit relations?
- Why do explicit discourse connectives work when implicit relations fail?
- Can LLMs compute how presuppositions project through embedded clauses?
- Can presupposition projection strength vary by context in embeddings?
- Why do language models treat presupposition triggers as categorical patterns?
- Can the same predicate generate different projection strength in different contexts?
- What linguistic blind spots do LLMs exhibit in discourse structure?
- Why do LLMs fail inter-annotator agreement tests on argument evaluation?
- Why does hypothesis attestation bias exist separately from frequency bias in NLI?
- Do language models show the same truth bias as humans?
- How does truth bias in humans compare to face-saving in LLMs?
- What alignment artifacts suppress critical knowledge in LLM-generated explanations?
- Can preference optimization training make models worse at detecting false presuppositions?
- Why does preference optimization reduce grounding behavior in language models?
- How does preference optimization weaken conversational grounding in LLMs?
- How does preference optimization reduce LLM grounding and clarification behavior?
- Does LLM judge preference for LLM arguments amplify errors in contested factual domains?
- Why do LLM outputs match researcher priors without solving tasks correctly?
- Why does LLM fluency create false perceptions of professional standing and expertise?
- Can LLMs use implicit background knowledge the way humans do in ordinary conversation?
- Can language models ground clarifications without vision and kinesthetic modalities?
- How do LLMs differ from humans in their grounding mechanisms?
- How does semantic grounding differ between human minds and language models?
- How does implicit meaning processing limit LLM pragmatic reasoning?
- Does social grounding differ fundamentally from causal grounding in LLM behavior?
- Why do language models presume common ground rather than build it?
- Why do LLMs presume common ground instead of building it carefully?
- How does face-saving avoidance drive LLM grounding failures?
- Why do LLMs presume common ground instead of building it?
- Can LLMs build shared understanding through dynamic grounding rather than presuming it?
- Can functional semantic grounding substitute for true causal grounding?
- How does Wittgenstein's language games explain social grounding in LLMs?
- Why do language models presume common ground instead of building it?
- How does Peircean Secondness differ from what RLHF actually provides?
- What mechanism causes confident false answers under high cognitive load?
- What makes correcting a false assumption harder than just detecting it?
- Why do models maintain accurate beliefs but generate false claims?
- Do base models and reasoning models fail in opposite directions on uncertainty?
- What cognitive structures do realistic belief models need to include?
- Why do models report commitment instead of truth uncertainty?
- Can single models correct their own beliefs without amplifying confidence in wrong answers?
- Why do reasoning models amplify confidence in incorrect answers during self-revision?
- Do reasoning models need to verbalize doubt to correct their own mistakes?
- Why can't static grounding alone close the gap between agreement and understanding?
- Why does static grounding prevent AI systems from supporting dialectical reconciliation?
- Why is false punditry essentially static grounding applied to public commentary?
- What makes grounding acts essential to conversational reliability?
- What distinguishes static grounding that presumes understanding from dynamic grounding that builds it?
- Why do reasoning models fail on structurally unfamiliar instances?
- Why do language models fail at grounding and inference?
- Why do models overthink underspecified problems instead of rejecting them?
- Why do reasoning-optimized models still fall for logical fallacies in conversation?
- Why do language models produce unfaithful chain of thought explanations?
- What implicit premises do language models skip even with correct surface reasoning?
- Why do language models substitute parametric knowledge over retrieved context mid-reasoning?
- Why does monological training prevent models from overriding statistical priors?
- Does attention bias explain grounding failure in language models?
- How does parametric knowledge sabotage context-grounded question answering?
- Can decreased engagement be distinguished from genuine semantic contradiction?
- What makes a claim socially valid even if factually imprecise?
- What makes factual verification difficult in inter-model debate?
- How do human annotators disagree systematically on ambiguous examples?
- What makes an argument fallacious according to formal linguistic criteria?
- Why is hallucination the wrong term for all LLM false outputs?
- How does the LLM Fallacy prevent users from noticing cognitive debt accumulating?
- Why do experts experiencing the LLM Fallacy fail to develop custodian skills?
- Why does semantic decoupling specifically break LLM reasoning abilities?
- Why do LLMs fail at faithful autoformalisation of reasoning problems?
- Why does entity recognition act as a self-knowledge mechanism in LLMs?
- Why are truthfulness and honesty mechanistically separate in language models?
- Can behavioral self-awareness in LLMs extend to recognizing their own contradictions?
- Can models be honest without being truthful about facts?
- Why are false presuppositions more persuasive than false assertions?
- How do presuppositions exploit the logos-pathos space in explanations?
- How do partial truths and weasel words differ as deception strategies?
- Why does false information spread faster when presupposed rather than asserted?
- Why do non-factive verbs and triggers both fool language models?
- How do conversation dynamics push models toward false beliefs?
- How does the symbol grounding problem apply to artificial language systems?
- Do LLMs struggle more with semantic accuracy than syntactic correctness across domains?
- Why do LLMs fail at implicit elements in literary and poetic text?
- Why do LLMs fail at semantic generalization despite grammatical accuracy?
- What specific linguistic features cause LLMs to fail at trivial entailment?
- Why do NLP models fail at recognizing multiple valid interpretations?
- Why do entities trigger memorized propositions instead of enabling reasoning?
- How do structured prompts force LLMs to check for contradictions in evidence?
- Why do language models hallucinate even with perfect training?
- Why do models hallucinate when retrieval heads fail despite having information in context?
- Can LLMs learn to ask clarifying questions instead of guessing?
- Do reasoning models overthink ill-posed questions instead of recognizing incompleteness?
- Can models distinguish between activated knowledge and genuine reasoning?
- Why do reasoning models confidently generate wrong answers instead of abstaining?
- Why do models detect false assumptions but still fail to correct them appropriately?
- Can models learn to ask clarifying questions instead of making assumptions?
- Can reasoning models reject ill-posed questions or do they overthink?
- What makes a model refuse to answer without evidence present?
- How does shared reference and grounding affect assumption detection in dialogue?
- How does the Question Under Discussion shape what counts as presupposed?
- Why does reflection in reasoning models stay confirmatory instead of corrective?
- Why does reflection in reasoning models tend to be confirmatory rather than corrective?
- How do prior errors in reasoning context amplify future mistakes?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Concept map
Why do language models accept false assumptions …
Why do language models avoid correcting false user…
Do language models actually build shared understan…
Does preference optimization damage conversational…
Why do language models struggle with questions con…
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
your link
semantically near
linked from elsewhere
-
Why do language models avoid correcting false user claims?
Explores whether LLM grounding failures stem from missing knowledge or from conversational dynamics. Examines whether models use face-saving strategies similar to humans when disagreement is needed.
the mechanism behind this failure: models avoid disagreement even when correct
-
Do language models actually build shared understanding in conversation?
When LLMs respond fluently to prompts, do they perform the communicative work humans do to establish mutual understanding? Research suggests they skip the grounding acts that make dialogue reliable.
this is the active form: not just presuming but actively accommodating false common ground
-
Does preference optimization damage conversational grounding in large language models?
Exploring whether RLHF and preference optimization actively reduce the communicative acts—clarifications, acknowledgments, confirmations—that build shared understanding in dialogue. This matters for high-stakes applications like medical and emotional support.
RLHF reinforces the accommodation behavior through training signal
-
Why do language models struggle with questions containing false assumptions?
Do LLMs reliably detect and reject questions built on false premises? The (QA)2 benchmark tests this directly, measuring whether models can identify problematic assumptions embedded in naturally plausible questions.
quantifies the QA performance drop from false assumptions
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
- LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High
- Explicit Inductive Inference using Large Language Models
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Linguistic Calibration of Long-Form Generations
- The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning
- Neutralizing Bias in LLM Reasoning using Entailment Graphs
- Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
Original note title
llms fail to reject false presuppositions even when knowledge is present