Can world models predict human action from physics alone?
Explores whether simulating only physical scenes is sufficient for predicting what people will do next, or whether explicit mental state modeling is necessary for accurate behavioral forecasting.
The paper's claim is that existing world-model formulations "merely answer a physical question: what/where it is, and how will it evolve," while human behavior is driven by hidden mental state — "what a person believes, wants, intends, feels, and considers socially permissible." A model that tracks the scene but not what each agent knows and believes about it "predicts the wrong action for the right-looking scene." Mental World Modeling (MWM) responds by making mental variables "core components of a world model rather than posthoc rationales." The conclusion reports consistent evidence across 8 modern LLM-based world models from two families: explicit mental world modeling is necessary for predicting human decisions, and the mental channel, the physical channel and their coupled transition are all required.
The mechanism is a single coupled world state. MWM maintains physical and mental state together, renders a target-specific partial observation (what a given agent can see and therefore know), and simulates how a candidate action updates both components at once. MENTIS, the paper's training-free and fully inspectable baseline, splits this into state parsing, target-observation generation, action decomposition, coupled physical and mental transition, and branch-level value evaluation. Two details carry the argument. The observation is partial and per target, so what an agent believes is derived from what that agent could perceive rather than read off the true scene. And the transition is coupled, so an action changes the material scene and the mental-social configuration together.
Against the neighbors, this note takes a scope item and turns it into a design commitment. What should a world model actually be designed to do? lists the social and mental worlds among the domains a general world model must simulate; MWM specifies what that requires for people, namely one state with both channels and a transition that updates both. Read against What five design choices compose a world model?, the coupling is a claim about representation and transition together, not either alone. That mapping is my reading, not the paper's. The failure it names, accurate on the visible variable and wrong on the hidden one, echoes Do foundation models learn world models or task-specific shortcuts?, though that note concerns physical trajectories and the excerpt makes no such comparison. On the theory-of-mind side, Do large language models genuinely simulate mental states? documents the behavioral gap. MWM is an architectural answer to it: mental state is made an explicit, inspectable part of the simulator, not something a model is hoped to infer implicitly.
The excerpt is silent on a lot. It gives no effect sizes, no testbed size or composition, and no scoring rule for "predicting human decisions." It does not name the two model families or say what the "human reference" is. The state taxonomy is mentioned but not shown, and the "necessity" result is stated without numbers. It also says the gains are largest on interpersonal scenes and hold across text, image and sounding-video inputs, without saying how large "largest" is. The one forward-looking result is that oracle interventions "localize most of the remaining gap to the human reference in transition simulation." Even that says nothing about which components were given oracle values. At the strength the excerpt supports, the priority order for future work is transition simulation, meaning how actions update belief and scene together, not state parsing or observation rendering. Whether the coupled formulation beats trained alternatives is untested in what is quoted, since MENTIS is presented as a baseline.
Inquiring lines that read this note 5
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do language models develop actual world models or merely task heuristics? What execution architectures enable agents to most effectively use tools? Why do persona simulations fail to predict authentic user behavior?Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
What should a world model actually be designed to do?
Current AI research treats world models as either video predictors or RL dynamics learners, but what if their real purpose is simulating actionable possibilities for decision-making rather than predicting next observations?
extends: takes its social and mental scope items and specifies the coupled state and transition they require
-
What five design choices compose a world model?
World models are often presented as monolithic systems, but they actually involve five distinct design decisions—data preparation, representation, reasoning architecture, training objective, and decision integration—that can each fail independently. Understanding this decomposition helps diagnose why world model proposals fall short.
complements: the coupled physical-mental state bears on the representation and architecture choices together
-
Do large language models genuinely simulate mental states?
This explores whether LLMs perform authentic theory of mind reasoning or rely on surface-level pattern matching. The distinction matters because evaluation format—multiple-choice versus open-ended—reveals very different capability levels.
complements: names the behavioral gap that MWM answers by making mental state explicit and inspectable
-
Do foundation models learn world models or task-specific shortcuts?
When transformer models predict sequences accurately, are they building genuine world models that capture underlying physics and logic? Or are they exploiting narrow patterns that fail under distribution shift?
contrasts: a physical-domain case of good surface accuracy without the underlying model, unlike this hidden-state case
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Mental World Modeling
- Can Language Models Serve as Text-Based World Simulators?
- Qwen-AgentWorld: Language World Models for General Agents
- Critiques of World Models
- Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
- PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues
- Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning
- What People Almost Did: Evaluating LLM Social Simulations Beyond Behavioral Fit
Original note title
world models that reason about people need coupled physical and mental state — the physical scene alone predicts the wrong action for the right-looking scene