Why might an AI agent do better with a record of what actually happened than with a forecast of what comes next?
What makes execution and memory proxies more useful than dynamics prediction alone?
This explores why a world model that hands an agent execution results and stored experience can be more useful than one that only predicts what the world will do next.
This explores why a world model that hands an agent execution results and stored experience can be more useful than one that only predicts what the world will do next. The corpus has a clear argument for this and evidence for its parts, but it has no head-to-head test against pure dynamics prediction.
The argument is about what the agent can do with the output. A dynamics model answers "what state comes next?", and the agent still has to turn that forecast into a decision. Should world models predict states or return agent feedback? proposes that world models instead return things an agent can act on directly: what happened when something was run, which past experiences apply, which skills fit, and whether an attempt passed verification. Dynamics is one of six forms on that list. Execution and memory replace a guess about the world with a fact or a recollection.
The execution side has the strongest evidence. Can execution harnesses lift model performance without retuning weights? shows that optimizing the execution system around frozen weights lifts accuracy across several models. The same runbook carried over to newer models unchanged, reaching 95.3% on GPT-5.6 and adding 5.4 points to DeepSeek-V4 Flash. So a lot of what an agent needs sits in the machinery that runs and checks its actions, not in a better internal prediction of the world. Can external state caches let models solve harder problems? takes a similar line with four levels of persistent state, and it separates harness failures from model failures. Its individual components were not ablated, so treat it as suggestive.
Memory and execution also work as a pair. In Should agent memory adapt dynamically based on execution feedback?, memory links form and get pruned according to whether execution worked, and this beat fixed retrieval on three benchmarks. Models are also poor places to keep experience internally. Can state-space models match transformers at copying and retrieval? shows fixed-size internal state provably limits copying and retrieval. Is long-context bottleneck really about memory or compute? finds the long-context problem is less raw capacity than the compute needed to turn old context into usable state. Handing the agent a pre-selected experience sidesteps both problems. There is a counterweight: Should agent memory live inside the model backbone? argues that external memory and the model backbone optimize separately and drift apart, so it builds memory into the backbone. What matters may be that the output is usable by the agent, not that the proxy sits outside the model.
There is also a practical benefit that the notes don't test against dynamics models: proxies can be inspected in pieces. How should we actually evaluate agent memory systems? splits memory into storage, extraction, retrieval and maintenance, and across 12 systems it shows which stage failed. How should we measure agent system performance beyond task success? shows identical success rates can hide large differences in efficiency and reliability. A dynamics model is judged mostly on how accurate its predictions are, which doesn't say whether the agent did better. Execution and memory proxies can be checked against what the agent actually did with them. That comparison is my inference from these notes and is not measured directly.
Sources 9 notes
Rather than predicting physical states, world models should function as Agent-Centric Interactive World Proxies that return execution outcomes, retrieved experiences, skills, and verification signals organized into six functional forms: dynamics, spatial, execution, memory/experience, skill, and reward/verification.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
Prime Agent organizes persistent state in four levels (weights, context, persistent REPL plus subagents, disk-backed history) to let models read and write addressable state beyond their instruction stream. The approach isolates harness failures from model failures and reported gains on ARC-AGI-3, though specific components remain unablated.
FluxMem demonstrates that adaptive memory topology—where links form, refine, and consolidate based on closed-loop execution feedback—consistently reaches state-of-the-art across three distinct benchmarks. Dynamic connectivity outperforms fixed retrieval by aligning abstraction and eliminating interference.
Two-layer transformers can copy exponentially long strings while state-space models are fundamentally limited by their fixed-size latent state. Empirically, transformers dramatically outperform SSMs at copying and context retrieval in both synthetic and pretrained settings.
Show all 9 sources
Research shows the bottleneck is not memory capacity but the compute required to consolidate evicted context into fast weights during offline sleep phases. Performance improves with more consolidation passes, following a test-time scaling pattern on harder reasoning tasks.
Metis demonstrates that agent memory can be implemented as a persistent state and autonomous procedures within the model backbone rather than external modules. This approach enables end-to-end training and avoids the decoupling failures where external memory and backbone optimize independently.
Decomposing memory into storage, extraction, retrieval, and maintenance stages exposes design trade-offs and failure modes that task-success metrics completely hide. Module-by-module evaluation across 12 systems shows which component actually failed rather than just whether the task succeeded.
Single task-success metrics obscure how agents achieve results across memory, context, and verification layers. Research shows identical success rates can mask enormous differences in efficiency, reliability, and deployment readiness—requiring harness-level benchmarks that measure trajectory, memory hygiene, and verification costs.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- From Model Scaling to System Scaling: Scaling the Harness in Agentic AI
- Are We Ready For An Agent-Native Memory System?
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization
- Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
- Useful Memories Become Faulty When Continuously Updated by LLMs
- GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents
- Metis: Memory Foundation Model
- Towards a Science of Scaling Agent Systems