SYNTHESIS NOTE
Topics›LLM Architecture›this note

Can world models predict human action from physics alone?

Explores whether simulating only physical scenes is sufficient for predicting what people will do next, or whether explicit mental state modeling is necessary for accurate behavioral forecasting.

Synthesis note · 2026-09-25 · sourced from LLM Architecture

The paper's claim is that existing world-model formulations "merely answer a physical question: what/where it is, and how will it evolve," while human behavior is driven by hidden mental state — "what a person believes, wants, intends, feels, and considers socially permissible." A model that tracks the scene but not what each agent knows and believes about it "predicts the wrong action for the right-looking scene." Mental World Modeling (MWM) responds by making mental variables "core components of a world model rather than posthoc rationales." The conclusion reports consistent evidence across 8 modern LLM-based world models from two families: explicit mental world modeling is necessary for predicting human decisions, and the mental channel, the physical channel and their coupled transition are all required.

The mechanism is a single coupled world state. MWM maintains physical and mental state together, renders a target-specific partial observation (what a given agent can see and therefore know), and simulates how a candidate action updates both components at once. MENTIS, the paper's training-free and fully inspectable baseline, splits this into state parsing, target-observation generation, action decomposition, coupled physical and mental transition, and branch-level value evaluation. Two details carry the argument. The observation is partial and per target, so what an agent believes is derived from what that agent could perceive rather than read off the true scene. And the transition is coupled, so an action changes the material scene and the mental-social configuration together.

Against the neighbors, this note takes a scope item and turns it into a design commitment. What should a world model actually be designed to do? lists the social and mental worlds among the domains a general world model must simulate; MWM specifies what that requires for people, namely one state with both channels and a transition that updates both. Read against What five design choices compose a world model?, the coupling is a claim about representation and transition together, not either alone. That mapping is my reading, not the paper's. The failure it names, accurate on the visible variable and wrong on the hidden one, echoes Do foundation models learn world models or task-specific shortcuts?, though that note concerns physical trajectories and the excerpt makes no such comparison. On the theory-of-mind side, Do large language models genuinely simulate mental states? documents the behavioral gap. MWM is an architectural answer to it: mental state is made an explicit, inspectable part of the simulator, not something a model is hoped to infer implicitly.

The excerpt is silent on a lot. It gives no effect sizes, no testbed size or composition, and no scoring rule for "predicting human decisions." It does not name the two model families or say what the "human reference" is. The state taxonomy is mentioned but not shown, and the "necessity" result is stated without numbers. It also says the gains are largest on interpersonal scenes and hold across text, image and sounding-video inputs, without saying how large "largest" is. The one forward-looking result is that oracle interventions "localize most of the remaining gap to the human reference in transition simulation." Even that says nothing about which components were given oracle values. At the strength the excerpt supports, the priority order for future work is transition simulation, meaning how actions update belief and scene together, not state parsing or observation rendering. Whether the coupled formulation beats trained alternatives is untested in what is quoted, since MENTIS is presented as a baseline.

Inquiring lines that read this note 5

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Do language models develop actual world models or merely task heuristics? What execution architectures enable agents to most effectively use tools? Why do persona simulations fail to predict authentic user behavior?

Related concepts in this collection 4

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
13 direct connections · 107 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

world models that reason about people need coupled physical and mental state — the physical scene alone predicts the wrong action for the right-looking scene