An AI can learn to use search engines by acting as its own search engine — until it hits facts it doesn't know.

This explores when LLMs generating their own fake search results (from internal knowledge) can train agents better — or cheaper — than letting them query a live search engine, and where that trick breaks down. The short version the corpus suggests: simulated search wins on cost and control during training, but real search wins whenever the task depends on facts the model doesn't already hold.

The strongest case for simulation is economic. Methods like ZeroSearch and SSRL show that an LLM can play the role of a search engine — generating plausible documents from its own weights — well enough that a 14B simulator matches or beats a real search API during reinforcement learning, with no per-query costs Can LLMs replace search engines during agent training?. The reason this works at all is that much of what RL search training teaches isn't *facts*, it's *process*: how to issue queries, read results, backtrack, and decide when to stop. A model learning to navigate can rehearse that loop against a simulated environment cheaply, and you can even degrade the simulator's quality on a curriculum to make training harder over time.

But simulation inherits a ceiling. The moment the task needs knowledge the model never absorbed — recent events, long-tail facts, anything past its training cutoff — the simulator can only hallucinate, and real search pulls ahead. DeepResearcher agents trained on live web search beat static memorized-knowledge models on knowledge-intensive tasks, and the corpus is blunt that the mechanism is retrieval, not reasoning: real search escapes the temporal bounds and lossy compression baked into training data Why do search agents beat memorized retrieval on hard questions?. This is the same trap that bounds any agent trained only on a curator's static dataset — competence is capped by what was imagined in advance, because the agent never touches a real environment Can agents learn beyond what their training data shows?.

There's a subtler risk too. RL search training tends to collapse behavioral diversity — policies converge on a few narrow reward-maximizing query strategies through the same entropy-collapse mechanism seen in reasoning models Does reinforcement learning squeeze exploration diversity in search agents?. A simulator that only reflects the model's existing knowledge could tighten that loop further, training an agent to search the way it already thinks rather than discovering new paths. This is why other corners of the corpus lean on *synthetic-but-grounded* environments instead of pure internal simulation: knowledge-graph random walks generate verifiable multi-hop questions with real structure underneath, training genuinely capable deep-search agents Can knowledge graphs generate training data for search agents?, and Stream-of-Search work shows that training on the messy full process — mistakes, backtracking and all — builds better internal search models than feeding only clean optimal trajectories Does training on messy search processes improve reasoning?.

The deeper reframe worth taking away: the field increasingly treats search itself as a *test-time compute axis*, where search budget scales answer quality on the same curve as reasoning tokens Does search budget scale like reasoning tokens for answer quality? How does test-time scaling work for individual research agents?. Through that lens, simulated vs. real search isn't a binary — it's a question of which axis you're training. Simulate to teach the *skill* of searching cheaply; reach for real search when the *substance* of the answer lives outside the model.