AI agents learn only from examples humans gave them, so can they learn beyond that without costly real-world practice?
Can agents escape training data distributions without expensive real-world interaction?
This explores whether agents trained on fixed expert data can learn things that data never showed them, using cheap substitutes such as simulated environments, their own actions, or memory, instead of costly live interaction.
This explores whether agents trained on fixed expert data can learn things that data never showed them, using cheap substitutes for live interaction. The corpus suggests the answer is mostly yes, but the escape routes come with limits. The problem is real: agents trained only on expert demonstrations never see their own failures, so their competence is capped by what the dataset's curators imagined rather than by the agent's capacity (Can agents learn beyond what their training data shows?). The obvious fix is to let agents practice in real environments, but that is the expensive part.
One route is to fake the environment with a language model. A language world model trained to predict what happens next across 10M+ trajectories was used to train agents, and they outperformed agents trained in the real environments on three benchmarks and transferred across seven domains (Can language models learn to simulate agent environments?). The same idea works at smaller scale for search: LLMs can generate search results from their own internal knowledge, and 14B simulators matched or beat real search engines as a training backend, with no API bills (Can LLMs replace search engines during agent training?). Both use a model as a cheap stand-in for the world, so the agent can fail and see consequences without paying for real interaction.
A second route treats the agent's own actions as the teacher. When an agent tries something and observes the resulting state, that consequence is a supervision signal, with no reward function or expert needed. Across eight environments this matched expert-dependent baselines with half the data and gave a better starting point for later RL (Can agents learn from their own actions without external rewards?). The corpus also covers a third route that leaves the weights alone. Agents can store verbal self-diagnoses of failures in episodic memory and improve across attempts (Can agents learn from failure without updating their weights?). Memory-based methods reached 87.88% on GAIA validation with no parameter updates (Can agents learn continuously from experience without updating weights?). Minecraft's VOYAGER agent keeps a growing library of executable skills, so new abilities are added without overwriting old ones (Can agents learn new skills without forgetting old ones?). These methods still touch a real environment, so they don't remove interaction. They make each interaction count for more.
The caution is that a simulator is itself built from training data. If the world model or search simulator only knows what its own data covered, the agent may get a wider curriculum without getting real ground truth. There is also evidence of where simulation breaks. LLMs look socially competent when one model plays every role, but fail systematically once agents hold private information, because the omniscient setup lets them skip the grounding work (Why do LLMs fail when simulating agents with private information?). So the cheap escapes work best where the environment is mostly public and consistent, such as search results, tool outputs and game states. They are less reliable where hidden information and other minds matter.