Should an AI decide for itself what to remember and forget, or is a separate manager better?
Does agent-side context control outperform external management on any task class?
This explores whether letting the agent itself decide what to keep, drop, or hand off beats a separate manager doing that for it, and whether that holds for any particular kind of task.
This explores whether letting the agent itself decide what to keep, drop, or hand off beats a separate manager doing that for it, and whether that holds for any particular kind of task. The corpus has no head-to-head test with the same task and model under both setups, so there is no clean 'agent-side wins on X' answer. It does hint at where the boundary sits.
The closest thing to an agent-side win is long-horizon search-style work. SearchSwarm trains a model to delegate subtasks and fold the summarized results back in. That active approach beats passive compression, and a 30B model matched much larger ones Can delegation teach models to manage context more actively?. The skill also transferred to single-agent tasks, which suggests the model learned disciplined decomposition rather than just orchestration. The comparison is against passive compression, though, not a trained external manager, so it isn't like-for-like. The Thread Inference Model is another agent-side case. The model structures its own reasoning as a tree of subtasks, and a rule prunes the KV cache. It stays accurate past context limits even while manipulating 90% of the cache, which lets one model replace a multi-agent setup Can recursive subtask trees overcome context window limits?. But the pruning rule is fixed, so this is a hybrid rather than pure agent judgment.
External management wins on a different axis, which is whether you can change the agent at all. AdaCoM trains a manager for frozen agents and finds the best policy depends on the agent, not the task. Stronger agents benefit from high-fidelity preservation, while weaker ones need aggressive compression Can an external manager handle context for frozen agents?. LLM Programs push external control further. An explicit algorithm owns the control flow and state, and each LLM call sees only the context relevant to its step. That helps when the model's capability or window is the bottleneck, and it makes each step debuggable Can algorithms control LLM reasoning better than LLMs alone?. For multi-turn workflows, agents failed from weak memory control, not missing knowledge. Transcript replay with no gating let errors and constraint drift accumulate. A bounded, schema-governed committed state that separates recall from permanent writes fixed it. That is control imposed by structure instead of left to the model's judgment Can agents fail from weak memory control rather than missing knowledge?.
So the corpus points to agent strength and trainability as the deciding factor, more than task class. If you can train the agent, teaching it to delegate and prune looks strong on long search and reasoning tasks. If the agent is frozen or weak, external structure earns its keep. Settling it would need trajectory-level measurement, because identical success rates can hide big differences in efficiency and memory hygiene How should we measure agent system performance beyond task success?.
Sources 6 notes
SearchSwarm shows that training models to delegate subtasks and integrate summarized results beats passive compression, with a 30B model matching much larger ones. Critically, the delegation skill transfers to single-agent tasks, suggesting it teaches disciplined decomposition and evidence grounding, not just orchestration.
The Thread Inference Model demonstrates that reasoning structured as recursive subtask trees with rule-based KV cache pruning sustains accurate reasoning beyond context limits, even when manipulating 90% of the cache. This enables single models to replace multi-agent systems by handling full recursive reasoning internally.
AdaCoM trains an external RL-based manager to prune and preserve context for frozen agents. The key finding: stronger agents benefit from high-fidelity preservation, while weaker agents need aggressive compression—optimal context management is agent-specific, not task-universal.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Agent performance degrades in long workflows because transcript replay and retrieval-based memory lack gating mechanisms. A bounded, schema-governed committed state that separates artifact recall from permanent memory write prevents error accumulation and constraint drift.
Show all 6 sources
Single task-success metrics obscure how agents achieve results across memory, context, and verification layers. Research shows identical success rates can mask enormous differences in efficiency, reliability, and deployment readiness—requiring harness-level benchmarks that measure trajectory, memory hygiene, and verification costs.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- From Model Scaling to System Scaling: Scaling the Harness in Agentic AI
- Learning Agent-Compatible Context Management for Long-Horizon Tasks
- Why Do Multi-agent LLM Systems Fail?
- SearchSwarm: Towards Delegation Intelligence in Agentic LLMs for Long-Horizon Deep Research
- LLMs Corrupt Your Documents When You Delegate
- Demystifying Agent Skills: Why They Work-Until They Don't
- Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
- Agent Workflow Memory