Can task state management alone improve long-horizon agent performance?
Does keeping verified task state outside the execution context, rather than embedded in a growing context window, help long-horizon agents succeed more often? This matters because it challenges assumptions about where bottlenecks actually occur.
The paper locates part of the long-horizon failure in where the harness keeps its bookkeeping. Per the abstract, existing harnesses "maintain task execution, task state, and completion assessment within a growing context," which makes the state "difficult to track" and lets "incorrect self-assessments" carry into later decisions. LongHorizon-Harness instead reformulates long-horizon execution as "a task-state management problem," holding the state explicitly outside execution and updating it only with facts independently verified from the environment. The reported results are Qwen 3.7-Plus rising from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench 2.1, and from 2.8% to 8.3% on OSWorld 2.0. The conclusion draws the moral that capability "is determined not only by the underlying model, but also by the harness."
The Manage-Execute-Audit (MEA) loop splits three jobs a single growing context was doing at once. A manager maintains the task state and picks the next subtask, a "fresh-context executor" performs it, and a "read-only auditor" verifies the resulting environment state before the next round. Only "independently verified outcomes" carry across rounds, so an executor's own claim of success does not become state by itself. A lightweight AgentAdapter lets model and harness backends be swapped "without modifying their native agent loops," and the conclusion reports improvements across hybrid GUI–CLI workflows, professional desktop tasks, pure command-line environments, and different model backbones.
Against the nearest notes, this extends Can delegation teach models to manage context more actively?. Both use fresh contexts for subtasks, but that note ends on the caution that a main agent integrates returned summaries it cannot audit, and the MEA loop answers with a read-only auditor that checks environment state instead of trusting the executor's report. That pairing is a reading of the two papers; the excerpt does not cite the delegation work. It contrasts with Can agents discover tools dynamically instead of pre-selecting them?, which keeps a whole-task perspective inside one autonomous stream, whereas MEA holds the whole-task view in the manager and gives each executor a narrow one. It also echoes the factoring in Why do planning and grounding pull against each other in agents?, another case of a harness-level split. And it adds a wrinkle to Does a single benchmark score actually predict agent readiness?: the same model differs by about 29 points on WeaveBench depending on the harness around it, so a reported score describes a model-plus-harness pair.
The excerpt gives only the abstract, one introduction passage and the conclusion. It reports no ablation, so it cannot say whether the gain comes from externalized state, fresh-context execution, or the audit step. It does not name the baseline harness, and it gives no run counts, task counts, or cost of the extra manager and auditor calls. It also does not say how the auditor verifies state where the environment offers little to check. The OSWorld 2.0 result is a large relative gain from a very low base, 8.3% in absolute terms. What the excerpt does support is narrower: harness design is a variable that moves long-horizon scores substantially for at least one model, and it belongs in the report beside the model name.
Inquiring lines that read this note 7
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can harness architecture and protocols provide agent reliability without model scaling? What mechanisms preserve shared understanding in evolving conversations? Can prompt-based context override biases that were embedded during pretraining? How do agent-learned skills transfer and improve across different tasks? How do surface patterns enable correct outputs but reduce robustness? What should agent evaluation prioritize to reveal reliable behavior?Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can delegation teach models to manage context more actively?
Does training models to decompose tasks and delegate to subagents—rather than passively compressing when context fills up—improve their ability to reason over long horizons? And does this skill transfer to single-agent work?
extends: fresh-context subtasks plus a read-only audit of environment state, rather than trusting returned summaries
-
Can agents discover tools dynamically instead of pre-selecting them?
Explore whether agents can find needed tools during execution rather than choosing from a fixed set upfront. This matters for long-horizon tasks where relevant tools cannot be known in advance.
contrasts: whole-task view in one stream there, held by a manager here
-
Does a single benchmark score actually predict agent readiness?
Single-axis benchmarks rank models by one capability—like task success—but ignore privacy, duration, operating mode, and ecosystem fit. Can one number really capture what matters for deployment?
qualifies: harness choice shifts one model's score, so scores describe model-plus-harness pairs
-
Why do planning and grounding pull against each other in agents?
Planning requires flexibility and error recovery while grounding demands action accuracy. Do these conflicting optimization requirements force a design choice about how to structure agent architectures?
parallels: another harness-level split of jobs that pull against each other in one policy
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
- Why Do Multi-agent LLM Systems Fail?
- Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
- AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
- Towards a Science of Scaling Agent Systems
- StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- interwhen: A Generalizable Framework for Steering Reasoning Models with Test-time Verification
Original note title
long-horizon execution is a task-state management problem — audited state kept outside execution lifted Qwen 3.7-Plus from 51.8 to 80.7 percent on WeaveBench