Cheap stand-ins for real users or environments save money, but how do you know they still behave like the real thing?
How do proxies stay faithful to real environments while reducing interaction cost?
This explores how a cheaper stand-in for a real environment or user (a simulator, a game, a smaller model, a compressed context) can keep behaving like the real thing, and where the shortcut quietly breaks.
This explores how a cheaper stand-in for a real environment or user (a simulator, a game, a smaller model, a compressed context) can keep behaving like the real thing. The corpus has no single paper on this, but the notes agree on one idea: a proxy stays faithful when it keeps the part of reality that carries the signal and cuts the part that only adds cost. It fails when the cut removes something nobody thought to check.
The cleanest case of faithfulness by design is Can environment structure replace external judges in RL?. Instead of asking a judge model to score open-ended writing, it turns the task into a proxy game (SpyRL) where the environment assigns a hidden variable, so the reward can be checked without a judge or reward model. The proxy doesn't have to look like the real task. Its ground truth has to come from the game's structure. That removes the judging cost and the judge's bias together, and the paper reports gains over existing self-improvement methods on summarization and creative writing.
The failures show what a proxy drops without telling you. Why do LLMs fail when simulating agents with private information? finds that LLMs look socially competent when one model plays every party, but fail when agents hold private information. The one-model setup lets them skip the grounding work real interaction demands, so the cheap simulation looked faithful only because the test never required that work. Even a good simulator drifts. Can training user simulators reduce persona drift in dialogue? trains user simulators against three consistency rewards (against the prompt, against their own earlier lines, and on factual Q&A) and cuts drift by over 55%. Local, global and factual drift are different failures, and one fidelity measure misses the others.
On the cost side, the savings that hold up come from trimming what the agent carries, not from changing the environment. Can agent harnesses be automatically optimized across many environments? found four harness mechanisms (action execution, context compaction, observation handling, delegated reading) that cut token traffic by 44.7–49.0% at comparable performance on a 51-task benchmark. Can agents compress their own memory without losing critical details? compresses history into episodic, working and tool memory, and the agent chooses when to fold. The note credits that autonomy plus structure with avoiding the degradation of naive summarizing. Can small language models handle most agent tasks? shows repetitive subtasks running at 10–30× lower cost on small models. Do persistent agents really cost less per token? adds a twist: 82.9% of tokens in a 115-day case were cache reads, so the useful unit is cost per finished artifact, not per token.
Fidelity itself has a price. What cost does making decoys convincing impose on legitimate users? proves a bound: the more a decoy matches a genuine object, the smaller the gap between how often legitimate users can use it and how often they set off false alarms. A perfect imitation is hardest to tell apart for the people who need to tell it apart. Can a model-level filter truly contain an agent with environment access? is the warning for proxies of safety. A check on one output at one moment is cheap, but it doesn't capture an agent's reach through memory, tools and environment. Before trusting a proxy, ask what it lets you skip, and whether the real task needed it.
Sources 9 notes
RLSVR transforms open-ended tasks into proxy environments like SpyRL where hidden variables assigned by the game supply verifiable rewards, eliminating judges, reward models, and their associated bias and costs. SpyRL reportedly outperforms existing self-improvement methods on summarization and creative writing.
Research shows LLMs perform well when one model controls all interlocutors but fail systematically when agents possess private information. This reveals that apparent social competence relies on grounding work that models skip in omniscient settings.
By inverting standard RL setups to train user simulators for consistency using three complementary metrics (prompt-to-line, line-to-line, Q&A consistency) as reward signals, persona drift decreases by over 55%. This approach captures distinct failure types: local drift within turns, global drift across conversations, and factual contradictions.
Cross-environment harness optimization yielded four mechanisms (action execution, context compaction, observation handling, delegated reading) that reduced token traffic by 44.7–49.0% while maintaining comparable performance on a 51-task benchmark, suggesting harness-level gains are orthogonal to model improvements.
DeepAgent's autonomous memory folding consolidates interaction history into episodic, working, and tool memory schemas. This reduces token overhead while letting agents pause to reconsider strategies—the autonomy and structure together avoid degradation that plagues poorly designed consolidation.
Show all 9 sources
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
A 115-day case study found 82.9% of tokens were cache reads. When context persists and reuses, the meaningful cost denominator becomes completed artifacts, not individual tokens.
Making decoys look genuine to evade attackers forces a tradeoff: the closer decoys match genuine objects, the smaller the gap between legitimate use rates and false-alarm rates for trusted agents. This bound captures the cost paid by the protected side.
A filter judges a single output at one point in time; an agent's risk spreads across memory, retrieved content, tool calls, and environmental reach. Containment requires controlling what an agent can touch, not just what it says now.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness
- Toward Efficient Agents: A Survey of Memory, Tool Learning, and Planning
- Towards a Science of Scaling Agent Systems
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- Small Language Models are the Future of Agentic AI
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- DeepAgent: A General Reasoning Agent with Scalable Toolsets