Do large language models make the same causal reasoning mistakes as humans?
Research on collider structures reveals whether LLMs share human biases in causal inference. This matters because if both fail identically, collaboration might reinforce rather than correct errors.
The collider structure C1 → E ← C2 (two independent causes with a shared effect) is a diagnostic test for normative causal reasoning. When you observe the effect E, observing one cause should lower your estimate of the other (explaining away). When E is absent, C1 and C2 should remain independent.
Humans systematically fail this test in characteristic ways:
- Weak explaining away: explaining away is present but weaker than normatively warranted
- Markov violations: treating supposedly independent causes as correlated even when no collider observation should create that correlation (a "rich-get-richer" associative bias)
The "Do LLMs Reason Causally Like Us?" paper (CLADDER dataset) finds that LLMs exhibit the same two biases in the same direction as humans. This is not the usual finding of LLM inferiority — it is a finding of human-like systematic error. LLMs are not categorically worse at causal reasoning; they err in the same direction.
This matters for several reasons. First, it undermines clean human-vs-LLM comparisons in causal reasoning tasks: if both fail in the same way, the relevant comparison shifts from "who is better" to "are the failure modes compatible." Second, it raises the question of mechanism: humans likely err due to the associative nature of pattern-matching; LLMs likely err for structurally related reasons (training on human text that exhibits the same biases). The shared error direction is evidence that Why do LLMs handle causal reasoning better than temporal reasoning? — the training data itself has these biases baked in.
Third, the finding has implications for high-stakes causal reasoning: medical diagnosis (collider structures appear in disease-symptom networks), legal reasoning (independent causes with shared outcomes), and policy analysis all involve collider-type structures. Human and LLM collaborators sharing the same biases may reinforce rather than correct each other's errors.
Inquiring lines that read this note 66
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What reasoning limitations and biases do language models systematically exhibit?- Do language models share the same cooperative truth-seeking rules as humans?
- Do language models exhibit the same causal biases that humans show?
- Do language models show the same truth bias as humans?
- Why do LLM outputs match researcher priors without solving tasks correctly?
- Why do review corpora contain biases that affect generated comparisons?
- Can LLMs reason through semantics without understanding causal mechanisms?
- Do causal rules enforce robustness that statistical patterns alone cannot maintain?
- Why do causal graphs alone fail to capture human reasoning processes?
- How do humans use associative reasoning without causal connections?
- Can causal models be extended to include non-causal cognition?
- What makes causal explanations stronger anxiety predictors than counterfactuals or dissonance?
- How does this motivational bias connect to LLMs' causal reasoning failures?
- Why do LLMs inherit causal biases from their training data?
- What are collider structures and why do they reveal reasoning errors?
- How might human-LLM teams reinforce each other's causal reasoning mistakes?
- Where do collider-type reasoning errors appear in real-world decisions?
- Why does LLM compression eliminate causal grounding in conceptual representations?
- What role does prediction error play in human event segmentation?
- Can causal belief networks extracted from interviews predict how people respond to policy changes?
- Why do causal reasoning directions succeed while temporal reasoning directions fail?
- How do causal belief networks extracted from interviews enable intervention reasoning?
- How does semantic association differ from mechanistic causal reasoning?
- Can external actions provide causal necessity that language models lack?
- How can extracted causal belief networks enable intervention simulation?
- Can a Reflect mechanism detect and revise failed causal predictions?
- How does causal structure avoid behaviorist limitations in LLM social simulation?
- What architectural changes would help LLMs distinguish causal relationships from temporal sequences?
- Do LLMs show stronger reasoning about causality than about temporal ordering?
- What computational methods most reliably establish causal evidence in AI mechanism discovery?
- Can LLMs distinguish causal mapping from superficial similarity matching?
- Can statistical correlations in LLMs function equivalently to human causal understanding?
- How does physical proximity to harm affect causal blame assignment?
- How do world models create indirect causal grounding without physical environment contact?
- Can language models develop world models that ground meaning in causal reality?
- Can functional semantic grounding substitute for true causal grounding?
- Why do LLMs fail at counterfactual reasoning despite factual knowledge?
- Why do LLMs reason fluently about causality but lack causal rigor?
- What makes a causal abstraction more transferable than a generic heuristic?
- What makes discourse structure different from mechanistic causal structure in traces?
- What computational features distinguish statistical from causal modes of reasoning?
- Do newer LLM generations create worse detector bias through increased linguistic divergence?
- Can tacit machine knowledge exist inside model weights as causal structure?
- What causal relationship exists between model weights and tacit knowledge in language models?
- What distinguishes a representational feature from a causally inert correlation?
- What prevents LLM representations from causally influencing generation outputs?
- How does completion bias in agents differ from other epistemic failure modes?
- How does proximity to an outcome affect causal blame in agent chains?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Why do LLMs handle causal reasoning better than temporal reasoning?
Exploring whether language models perform asymmetrically on different discourse relations and what training data patterns might explain the gap between causal and temporal reasoning abilities.
the training-data explanation for why LLMs inherit human causal biases; the collider finding is a specific manifestation
-
Do LLMs generalize moral reasoning by meaning or surface form?
When moral scenarios are reworded to reverse their meaning while keeping similar language, do LLMs recognize the semantic shift? This tests whether LLMs actually understand moral concepts or reproduce training distribution patterns.
parallel insight: LLM errors track surface statistical regularities in training data, not normative structure
-
Do foundation models learn world models or task-specific shortcuts?
When transformer models predict sequences accurately, are they building genuine world models that capture underlying physics and logic? Or are they exploiting narrow patterns that fail under distribution shift?
collider bias is one instance: surface associative patterns override normative causal structure
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Do Large Language Models Reason Causally Like Us? Even Better?
- Premise Order Matters in Reasoning with Large Language Models
- Large Causal Models From Large Language Models
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
- LLMs can implicitly learn from mistakes in-context
- Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
- What Do Large Language Models Know? Tacit Knowledge as a Potential Causal-Explanatory Structure
Original note title
llms exhibit human-like causal biases — weak explaining away and markov violations in collider networks