Before an AI agent acts, can a model predict what will happen? It seems to work best where outcomes are text.
What domains let world models predict execution outcomes accurately enough?
This explores which kinds of environments, such as running code, using tools or dealing with people, are predictable enough that a model can forecast what will happen when an agent acts, before the agent actually does it.
The corpus has no ranked list of domains where this works, so I'll say that up front. It does have three useful clues: where it has worked, what makes a domain predictable, and where it breaks.
The strongest direct evidence is for digital environments where the outcome of an action is itself text. Qwen-AgentWorld trained a language model to predict the next state from more than 10 million agent trajectories. Agents trained against that simulated world beat agents trained in real environments on three benchmarks, and the approach transferred across seven domains (Can language models learn to simulate agent environments?). This fits a broader argument that a world model should hand agents usable feedback, meaning execution outcomes, retrieved experience and verification signals, rather than reconstruct physical states (Should world models predict states or return agent feedback?). What links these is that the world answers back in a form the model can read and reproduce.
The corpus doesn't test this directly, but a lateral clue comes from a different question, which domains suit autonomous research. Four properties decide it: an immediate scalar metric, modular structure, fast iteration cycles and version control. When one is missing, the domain resists automation however strong the model is, because the bottleneck is environmental structure, not model power (What makes a research domain suitable for autonomous optimization?). My guess is that the same properties make outcomes easier to predict. Clear scores and quick, repeatable feedback are also what long-horizon agents lean on: repeated benchmark-edit-incorporate loops predicted success better than initial quality did (What predicts success in ultra-long-horizon agent tasks?). Even in forecasting, which is messier, structure matters more than raw model strength. LLMs predict time series much better when the workflow separates numerical reasoning from contextual reasoning (Can LLMs actually forecast time series better than we think?).
Domains involving people are where prediction breaks down. Tracking only the physical scene gives wrong predictions of human action even when the scene itself is right. Accuracy needed beliefs, wants and intentions modeled explicitly and coupled to the physical state (Can world models predict human action from physics alone?). So an execution outcome that depends on a person's decision needs a different kind of world model than one that depends on a compiler.
The last caveat is about what accurate means. Models trained on orbital mechanics and games predicted well while learning task-specific shortcuts rather than a unified picture of the world, and fine-tuning exposed laws that made no sense (Do foundation models learn world models or task-specific shortcuts?). A high score in a domain can therefore mean the shortcut is easy there, not that the world is well understood. That is why the corpus keeps pushing world models toward simulating interventions and counterfactuals (What makes a world model actually useful for reasoning?). One practical way to decide when to trust a prediction is to ground confidence in a stored history of past outcomes (Can past performance predict when a model will be right?).
Sources 9 notes
Qwen-AgentWorld demonstrates that native language world models trained via next-state prediction on 10M+ trajectories outperform real-environment training on three benchmarks and transfer across seven domains, positioning next-state prediction as a foundation objective for agents.
Rather than predicting physical states, world models should function as Agent-Centric Interactive World Proxies that return execution outcomes, retrieved experiences, skills, and verification signals organized into six functional forms: dynamics, spatial, execution, memory/experience, skill, and reward/verification.
Autonomous research pipelines require immediate scalar metrics, modular architecture, fast iteration cycles, and version control. Domains lacking any property resist autoresearch regardless of LLM capability, because the bottleneck is environmental structure, not model power.
Across 17 frontier models on 36 expert-curated optimization tasks, repeated benchmark-edit-incorporate cycles within a wall-clock budget proved the dominant success predictor. Most models terminated early or burned budget unproductively; Claude Opus 4.6 stood out as persistent.
LLMs have stronger intrinsic forecasting ability than recognized, but only when workflows separate numerical reasoning from contextual reasoning. Monolithic prompting obscures this capability; structured decomposition surfaces it.
Show all 9 sources
Research across eight LLM-based world models shows that tracking only the physical scene leads to wrong action predictions even when the scene looks correct. Mental World Modeling makes beliefs, wants, and intentions explicit state components coupled to physical simulation, and all three elements are required for accurate human decision prediction.
Inductive bias probes show transformers trained on orbital mechanics and games learn predictive patterns, not unified world structure. Fine-tuning reveals nonsensical, slice-dependent laws; circuit analysis shows arithmetic relies on range-matching heuristics, not algorithms.
Research shows LLMs may achieve high prediction accuracy through task-specific heuristics without developing coherent generative models of how the world works. True world models must enable reasoning about interventions and counterfactuals, not surface regularities.
XConf matches ten-sample self-consistency at a tenth of the cost by retrieving the model's past episodes with similar confidence levels and reading their historical success rates. Ablations show the signal depends entirely on stored outcomes, not on the retrieval prompt itself.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Qwen-AgentWorld: Language World Models for General Agents
- Can Language Models Serve as Text-Based World Simulators?
- Mental World Modeling
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
- What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models
- AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
- Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning