What's the difference between representing world facts and generating world mechanisms?
This explores the gap between an AI that can store accurate facts about the world and one that can simulate how the world actually works — the difference between a coherent map of what is and a runnable engine of what would happen if.
This explores the gap between representing world facts and generating world mechanisms — between an LLM that holds accurate descriptions of the world and one that can run a model of how the world changes. The corpus draws this line sharply. LLMs are very good at the first: they extract coherent factual structure from text and can lay out what is true. But the same probing evidence shows they stumble at the second — reasoning that requires counterfactual manipulation or causal intervention — because they lean on task-specific heuristics rather than a generative model of how things work Do LLMs actually have world models or just facts?. The word "world model" itself hides this ambiguity: it can mean a tidy representation of facts, or a machine that simulates consequences, and these are not the same achievement.
The distinction matters because high prediction accuracy can be a mirage. A system can nail next-token or next-observation predictions through surface regularities while having no engine underneath that lets it ask "what if I intervened here?" What makes a world model actually useful for reasoning?. That's why one line of work reframes the entire goal of a world model away from passive prediction and toward simulating actionable possibility spaces — physical, social, counterfactual, embodied — grounded in an agent's decisions rather than in forecasting the next frame What should a world model actually be designed to do?. Generating mechanisms means being able to run hypotheticals; representing facts only means being able to recite them.
There's a deeper methodological echo here. Studying whether a model truly has mechanisms — not just correlated features — requires more than reading off its representations. Representational analysis alone finds correlations without causation; you have to intervene causally to confirm a mechanism is actually doing work Can we understand LLM mechanisms with only representational analysis?. So the fact/mechanism split shows up twice: once in what the model possesses (facts vs. a causal engine), and again in how we'd even verify the difference (correlation vs. intervention). And it surfaces yet again in pretraining itself: factual recall depends on narrow, document-specific memorization, while reasoning that generalizes rides on broad procedural knowledge spread across many sources — two different things the model learns in two different ways Does procedural knowledge drive reasoning more than factual retrieval?.
What you didn't know you wanted to know: this isn't a single yes/no verdict but a design space. One framing decomposes any world model into five separable choices — data, latent representation, reasoning architecture, training objective, and how it plugs into decisions — and the point is that each can quietly misalign with the others, so "does the model have a world model?" is the wrong question; "which of these five is failing?" is the right one What five design choices compose a world model?. There's even a counterweight to the skeptics: by extracting regularities from text written by causally grounded humans, LLMs may acquire a kind of indirect causal grounding — real but mediated, with gaps that block real-time verification and updating Can large language models develop genuine world models without direct environmental contact?. Representing facts is having the map; generating mechanisms is being able to redraw the map when the territory changes.
Sources 7 notes
LLMs coherently represent factual world structure from text but fail at mechanistic reasoning requiring counterfactual manipulation or causal intervention. Probe evidence shows they rely on task-specific heuristics rather than generative models of how the world works.
Research shows LLMs may achieve high prediction accuracy through task-specific heuristics without developing coherent generative models of how the world works. True world models must enable reasoning about interventions and counterfactuals, not surface regularities.
Drawing on hypothetical thinking in psychology, world models are most useful when designed to simulate all actionable possibility spaces—physical, embodied, emotional, social, mental, counterfactual, and evolutionary—grounded in agent decision-making rather than passive prediction.
Representational analysis alone identifies correlations without causation; causal analysis alone shows behavioral effects without explaining them. Only paired methods—locating candidate features representationally, then verifying causally—produce complete mechanistic claims.
Analysis of 5 million pretraining documents shows reasoning relies on broad, transferable procedural knowledge from diverse sources, unlike factual recall which depends on narrow, document-specific memorization of target facts.
World model design comprises five distinct dimensions: data preparation, latent representation, reasoning architecture, training objective, and decision-system integration. Each can misalign with the others, and treating them as a single problem obscures where failures originate and prevents proper evaluation.
LLMs form structured world representations by extracting regularities from training data produced by causally grounded humans. This constitutes indirect causal grounding mediated through text, though the chain has gaps that limit real-time verification and model updating.