Do large language models embody genuinely new modes of understanding?
Mitchell and Krakauer explore whether LLMs represent a fundamentally different kind of intelligence—one based on statistical correlation rather than causal reasoning—and whether this requires rethinking how we define understanding itself.
Mitchell and Krakauer survey a split in the AI research community over whether large language models understand language. One camp holds they do: an AI researcher found LaMDA convincing enough to say it "in a very real sense understand[s] a wide range of concepts" and is "making strides towards consciousness," and another group argues LLMs "likely capture important aspects of meaning" by "approximat[ing] a compelling account of human cognition in which meaning arises from conceptual role." The opposing camp argues LLMs "cannot possess understanding because they have no experience or mental models of the world," calling them "doomed to a shallow understanding that will never approximate the full-bodied thinking we see in humans" no matter how long they train. Rather than picking a side, the authors conclude that LLMs may have produced "new modes of understanding, most likely new species in a larger zoo of related concepts" — comparing the kind of "alien" intuition AlphaZero and AlphaFold bring to chess and protein structure.
Their reasoning rests on a contrast between "mental models that rely on statistical correlations and those that rely on causal mechanisms." Human understanding, in their account, is built from concepts — "internal mental models of external categories, situations, and events" organized into "coherent, hierarchical sets of relations... that include underlying causal knowledge" — which support compositional and counterfactual reasoning and active intervention on the world. LLMs, they argue, have "extraordinary formal linguistic competence" (fluent, grammatical output) without the "conceptual understanding needed for humanlike functional language abilities": "understanding a tickle is to map a word to a sensation, not to another word." But instead of resolving the debate, they pose it as three open questions — is attributing understanding to LLMs a category error; do LLMs build real concept-based models despite lacking a body; or can their statistical correlations be "functionally equivalent to human understanding," perhaps even "forms of higher-order logic that humans are incapable of accessing" — and answer with pluralism: large-scale statistical models will win where "historically encoded knowledge" is abundant and performance is at a premium, while causal human intelligence will win where data is scarce and mechanisms are strong.
This reframes the debate at a more general level than most of the library's later entries. Do language models understand in fundamentally different ways? replaces Mitchell and Krakauer's informal three-way split with circuit-level evidence for a tiered, non-binary structure inside a single model — essentially an empirical answer to the "do they build real concepts" branch of the authors' three questions. What misconceptions hide in how we describe large language models? later formalizes the same "both camps see something real" structure into four explicit distinctions, suggesting the diagnosis here predates and anticipates that framework by several years. Do foundation models learn world models or task-specific shortcuts? supplies direct evidence for the no-causal-model side of the debate the authors describe. Are we underestimating human minds while debating machine minds? runs the same mind-attribution question in the opposite direction.
The excerpt is a position piece, not an experiment: it adjudicates no study and gives no criterion for telling "functionally equivalent" understanding apart from mere fluent output, and the AlphaZero/AlphaFold analogy to "alien" intuition is asserted rather than tested. The authors explicitly leave open which of their three framing questions is correct. What follows, at the strength a survey piece supports, is that "does this LLM understand" is underspecified until paired with a stated tier or criterion — a gap later mechanistic work tries to close empirically, not a question this paper settles.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do AI explanations represent actual model reasoning or constructed post-hoc narratives? Why do language models systematically fail at causal reasoning? Does reasoning architecture matter more than model scale?Related concepts in this collection 7
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do language models understand in fundamentally different ways?
Does mechanistic evidence reveal distinct tiers of understanding in LLMs—from concept recognition to factual knowledge to principled reasoning? And do these tiers coexist rather than replace each other?
replaces the authors' informal three-way split with circuit-level evidence for coexisting, non-binary tiers of understanding
-
What misconceptions hide in how we describe large language models?
Both deflationary framings like "just autocomplete" and anthropomorphic claims about emerging agency capture something real about LLMs, but each may overextend its truth. What distinctions help separate genuine features from overreach?
later formalizes the same two-camps-both-partly-right structure into four explicit distinctions
-
Do foundation models learn world models or task-specific shortcuts?
When transformer models predict sequences accurately, are they building genuine world models that capture underlying physics and logic? Or are they exploiting narrow patterns that fail under distribution shift?
supplies empirical evidence for the no-causal-model side of the debate surveyed here
-
Are we underestimating human minds while debating machine minds?
Public AI discourse focuses on whether machines have too much attributed mind, but what if the real risk is humans coming to see themselves as mere language models? This explores the neglected inverse problem.
reframes the same mind-attribution debate in the opposite direction, toward under-attributing mind to humans
-
Can LLMs understand concepts they cannot apply?
Explores whether large language models can correctly explain ideas while simultaneously failing to use them—and whether that combination reveals something fundamentally different from ordinary mistakes.
Contradicts: potemkin failures show structural incoherence, undermining the claim that LLMs exhibit a coherent new mode of understanding
-
Do large language models reason symbolically or semantically?
Can LLMs follow explicit logical rules when those rules contradict their training knowledge? Testing whether reasoning operates independently of semantic associations reveals what computational mechanisms actually drive LLM multi-step inference.
Evidence for: LLMs' reliance on semantic association over symbolic rules supports the non-causal mode of understanding
-
Do LLM improvements reflect reasoning gains or corpus shifts?
When large language models improve on previously failed tasks, does this show they've learned to reason better, or are they simply reflecting changes in human-written text they train on? Understanding this matters for assessing what LLMs actually know.
Contradicts: gains trace to shifting human discourse, not LLM reasoning, undercutting the claim of a new machine understanding
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- The Debate Over Understanding in AI's Large Language Models
- Word Meanings in Transformer Language Models
- From Human to Machine Psychology: A Conceptual Framework for Understanding Well-Being in Large Language Models
- The Missing Layer of AGI: From Pattern Alchemy to Coordination Physics
- Beyond Hallucinations: The Illusion of Understanding in Large Language Models
- Language Models are Pragmatic Speakers
- The Homogenizing Effect of Large Language Models on Human Expression and Thought
- Large Language Models and Scientific Discourse: Where's the Intelligence?
Original note title
Mitchell and Krakauer argue LLMs embody a new non-causal mode of understanding that needs an extended science of intelligence