Quo Vadis, World Modeling?
Continually improving agents require dynamic interaction feedback beyond static supervision, yet direct real-environment interaction is costly, slow, unsafe, and hard to parallelize. World modeling offers a natural intermediate proxy that allows agents to query lower-cost, more controllable feedback before committing to real actions. Classical world models instantiate this proxy primarily through future physical-state prediction, a formulation useful yet narrow for agents that require actionable feedback beyond raw state transitions. In this work, we conceptualize Agent-Centric Interactive World Proxies, shifting the fundamental paradigm from physical state transitions to agent-usable information transitions, such as execution outcomes, retrieved experiences or skills, and verification signals, broadening the scope of world modeling to provide versatile feedback for continually improving agents. To systematically map this design space, we organize world proxies into six functional forms based on their feedback modalities: dynamics, spatial, execution, memory/experience, skill, and reward/verification proxies, which together characterize the primary ways world modeling serves agent improvement.
Introduction. A truly capable agent does more than finish the task in front of it. It explores the unfamiliar, draws feedback from the world, and turns every success and failure into momentum for the next attempt [157, 203]. Capability of this kind is not granted once at training time; it is earned, again and again, through interaction. Yet most agents today are still taught the way students cram for an exam: from static, offline data such as expert trajectories, human annotations, or supervised fine-tuning corpora [10, 122]. Such data captures patterns that already exist, but it cannot answer back when the agent strikes out on its own. The agent’s competence is therefore bounded by the distribution it was trained on, and the new information that only active trial and error can reveal stays out of reach [155]. This is the ceiling that genuine self-improvement must break through. How can an agent move beyond static supervision, gather useful feedback through active interaction, and use it to improve without end?
Discussion / Conclusion. We opened with a question, so let us close with an answer. For most of its history, world modeling has been pursued as the art of predicting the world: given a state and an action, render the next frame as faithfully as possible [36]. This article has argued for a quieter but consequential shift, from predicting the world to serving the agent. Once the goal becomes continual improvement, the right object is no longer a state predictor but an Agent-Centric World Proxy: an environment-grounded mechanism that returns the information transition an agent needs, whether a future state, rendered view, execution result, retrieved memory or skill, or verdict on a plan. That reframing organized the rest of the story. We saw why real environments alone cannot carry continual improvement, what changes when physical state transitions become interactive information transitions, how proxies empower agents across L.1 inference-time guidance, So, quo vadis?
Lines of inquiry this paper opens 24
Research framings built by reading the notes related to this paper — the questions it feeds into.
How do multi-agent systems achieve genuine cooperation and reasoning?- Do explicit reward structures enable AI agent cooperation that open-ended interaction cannot?
- Do dynamic environments enable different kinds of agent-environment coevolution?
- How do agents ground their judgments in evidence instead of pattern matching?
- How does co-player diversity force agents to develop general adaptation?
- How do controllable simulators compare to population-level agent simulation approaches?
- How much does agent performance depend on demonstration quantity versus curation quality?
- What domain properties determine whether causal rules transfer to new agents?
- How does real tool integration change what agents learn compared to simulated tools?
- Can agentic reasoning outperform rigid rule-based systems for skill refinement?
- What capabilities can emerge from self-modification that the original agent lacked?
- Can next-state supervision work across different agent interaction types like conversations and tool calls?
- What happens when agents interact with environments and learn from their own mistakes?