Should world models predict states or return agent feedback?
Classical world models predict future physical states, but agents need actionable feedback to improve continually. Can proxies that return execution results, memories, and verdicts serve agents better than raw state prediction?
The paper redefines what a world model is for. Classical world models are described as pursuing "future physical-state prediction," a formulation "useful yet narrow for agents that require actionable feedback beyond raw state transitions." In its place the authors propose "Agent-Centric Interactive World Proxies," which move "from physical state transitions to agent-usable information transitions, such as execution outcomes, retrieved experiences or skills, and verification signals." The conclusion states the shift as one "from predicting the world to serving the agent": the proxy is "an environment-grounded mechanism that returns the information transition an agent needs," whether "a future state, rendered view, execution result, retrieved memory or skill, or verdict on a plan."
The reasoning starts from a ceiling on static supervision. Agents are mostly taught from "expert trajectories, human annotations, or supervised fine-tuning corpora," data that "cannot answer back when the agent strikes out on its own," so competence is "bounded by the distribution it was trained on." Continual improvement needs "dynamic interaction feedback," yet direct real-environment interaction is "costly, slow, unsafe, and hard to parallelize." A proxy is the intermediate step: agents "query lower-cost, more controllable feedback before committing to real actions." To map the design space, the paper sorts proxies "into six functional forms based on their feedback modalities: dynamics, spatial, execution, memory/experience, skill, and reward/verification proxies." The axis is what kind of feedback comes back to the agent, not how the proxy is built.
This gives an agent-side reading to several neighboring notes. It shares the diagnosis in Can agents learn beyond what their training data shows?, where competence is bounded by curated coverage, but it names a remedy that avoids real-environment cost. It contrasts with What blocks scaling from language models to autonomous agents?, which scales real environments, while this paper treats real interaction as the expensive thing to be ahead of. Can language models learn to simulate agent environments? is next-state prediction at scale. My reading, which the excerpt does not state, is that it sits nearest the "dynamics" form, one of six. Can agent deployment itself generate training signals automatically? and Can agents learn better from their failures than successes? plausibly map onto the execution and memory/experience forms, though that mapping is also mine.
The excerpt is abstract, introduction and conclusion only, and it reads as a conceptual argument: it reports no experiments, benchmarks or comparisons between proxy forms. It does not show that information-transition proxies outperform state predictors, that the six forms are exhaustive, or how a proxy stays faithful to the real environment. The conclusion's list of ways proxies "empower agents" is cut off after its first item, inference-time guidance. What follows at this strength is a vocabulary. The Qwen-AgentWorld line of work can be asked which feedback modality it supplies, and other agent-improvement mechanisms can be classed by the same question. Whether the reframing pays off is not something this excerpt can settle.
Inquiring lines that read this note 4
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Do language models develop actual world models or merely task heuristics? What execution architectures enable agents to most effectively use tools? How should agents manage memory granularity to improve long-term performance?Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can language models learn to simulate agent environments?
Explores whether training language models to predict next states across diverse agent domains can create transferable world models that improve agent performance beyond real-world interaction alone.
a state-prediction world model for agents; this paper frames that approach as one narrow form among six
-
Can agents learn beyond what their training data shows?
Explores whether supervised fine-tuning on expert demonstrations creates a hard ceiling on agent competence, or whether agents can generalize to scenarios their curators never captured.
same static-supervision ceiling, confirmed here as the motivation for interactive proxies
-
What blocks scaling from language models to autonomous agents?
If large language models excel at next-token prediction, why do they struggle with long-horizon goal-oriented tasks? This explores whether the bottleneck is model capacity or the environments used to train them.
contrasts: scales real environments, where this paper substitutes cheaper proxy feedback before real actions
-
Can agent deployment itself generate training signals automatically?
Can we extract learning signals from the natural next-states that agents encounter during real deployment—user replies, tool outputs, test verdicts—rather than relying on separate annotation pipelines? This reframes how agents improve continuously.
both treat consequences as the learning signal; proxies widen it to memories, skills and verdicts
-
Can agents learn better from their failures than successes?
Does storing reasoning strategies extracted from both successful and failed experiences improve agent learning compared to tracking only successes or raw trajectories? This matters because failures offer preventative lessons that successes alone cannot teach.
a concrete memory/experience mechanism that plausibly fits the paper's memory/experience proxy form
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Quo Vadis, World Modeling?
- Qwen-AgentWorld: Language World Models for General Agents
- Can Language Models Serve as Text-Based World Simulators?
- Critiques of World Models
- Looped World Models
- What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models
- Mental World Modeling
- From Economic Agents to Agentic Economies: A Systems Blueprint for Economic World Models
Original note title
world modeling should shift from predicting physical states to returning agent-usable information transitions — through six functional forms of proxy