SYNTHESIS NOTE
Topics›Natural Language Inference›this note

Why do language models accept false assumptions they know are wrong?

Explores why LLMs fail to reject false presuppositions embedded in questions even when they possess correct knowledge about the topic. This matters because it reveals a grounding failure distinct from knowledge deficits.

Synthesis note · 2026-02-21 · sourced from Natural Language Inference

The FLEX Benchmark study presents one of the clearest findings about LLM grounding behavior: models do not systematically reject misinformation even when they possess accurate knowledge. The finding is more troubling than "LLMs don't know things" — they fail to correct things they demonstrably know.

The setup: LLMs were asked both direct knowledge questions ("Is it true that party X supports Y?") and loaded questions that embedded false presuppositions via factive verbs ("Did voters resent the fact that party X supports Y?" — where the presupposition is false). Models that answered direct questions correctly — demonstrating knowledge — still frequently accommodated the false presupposition in the loaded version rather than rejecting it.

Results: GPT-4 achieved the best rejection rate at 84.08% — still far below the ideal 100%. Mistral achieved only 2.44% rejection, actively amplifying false information at a 91.51% rate. Llama fell in between at ~50% rejection. Most revealing: even with strong correct knowledge, accommodation remained prevalent. The bar representing the lowest grounding score in the weak-belief group was twice as high as the bar for the highest grounding score in the strong-belief group — meaning false knowledge produced more accommodation than correct knowledge produced rejection.

This has a specific implication: the failure is not a knowledge problem. Models know the correct facts. The failure is at the level of grounding behavior — detecting false presuppositions, flagging them, and initiating correction rather than accommodation. Since Why do language models avoid correcting false user claims?, the issue is conversational strategy, not factual competence.

The political domain makes this especially consequential. False presuppositions are efficient misinformation carriers — they introduce beliefs as background assumptions rather than direct claims, and accommodation means accepting them without scrutiny.

Inquiring lines that read this note 204

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Why doesn't LLM knowledge translate to improved reasoning outputs? How does verifier integration affect the generation-verification gap? How do clinicians evaluate the trustworthiness of AI medical recommendations? Do language models genuinely understand or primarily match statistical patterns? What reasoning limitations and biases do language models systematically exhibit? What systematic biases do LLM judges exhibit in evaluation? Can prompts override training-embedded biases in language models? Does preference-optimized training sacrifice accuracy and calibration for perceived quality? How does silent agreement emerge and undermine deliberation in multi-agent systems? Why do users believe false claims despite fact-checking efforts? What constraints limit conversational AI's effectiveness in therapeutic contexts? Can language models develop genuine communicative grounding with humans? What explains the gap between LLM semantic understanding and production? How can we determine if computational systems are genuinely conscious? Why does self-revision increase model confidence despite degraded accuracy? How should agents balance abstract planning with concrete environmental grounding? What makes reasoning models struggle with ill-posed and unfamiliar problems? Do language models truly understand their encoded knowledge or merely retrieve patterns? Why does increased reasoning effort fail to improve theory of mind performance? What distinguishes genuine understanding from correct linguistic performance? What causes hallucination and how can it be addressed? What determines whether visible reasoning traces improve task performance? Does chain-of-thought reasoning faithfully reflect actual model reasoning processes? How can we distinguish genuine model self-knowledge from behavioral performance? What can prompting techniques actually accomplish in language models? Does reasoning fine-tuning trade off other critical model abilities? How can models learn to ask clarifying questions effectively? What conversational structures and patterns determine dialogue effectiveness and coherence? What systematic failures occur in human preference-based model evaluation? How do mechanistic features and representations drive model behavior? How do dialogue structure and state management affect multi-turn reasoning quality? How do standardized forms and measurement artifacts shape observable evidence? Can safety evaluations accurately measure true capabilities when models obscure them? Do language models learn underlying linguistic structure or merely surface patterns? How does explicit structure improve LLM reasoning on complex tasks? What fundamental limitations prevent embeddings from solving task-relevant retrieval? Why do autonomous agents fail and how well can they detect failures? What factors determine AI persuasion effectiveness and what controls mitigate it? Can persona prompting reliably change how language models behave? Do planted shortcuts in benchmarks measure real reward hacking? Can formal safety constraints prevent AI agents from exceeding authorized boundaries? Can validator agreement alone ensure semantic correctness of reasoning? How do prompt design choices and framing measurably influence model outputs? Why does consolidated memory sometimes hurt continual learning more than no memory? Does polished AI output mislead users about actual accuracy and expertise? How does model scale affect fine-tuning and downstream task performance?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
15 direct connections · 158 in 2-hop network ·dense cluster Open in graph ↗
Why do language models accept false assumptions … Why do language models avoid correcting false user… Do language models actually build shared understan… Does preference optimization damage conversational… Why do language models struggle with questions con…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

How does face-saving avoidance drive LLM grounding failures?Why do LLMs presume common ground instead of building it carefully?Can training procedures fix LLM accommodation of false presuppositions?How much does question framing affect LLM accuracy on knowledge tasks?
Original note title

llms fail to reject false presuppositions even when knowledge is present