Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories

Agents built around large language models continually accumulate interaction trajectories during deployment, yet their behavior typically remains fixed. Beyond updating model weights, these trajectories can improve the agent harness that constructs context, mediates tools, validates actions, and recovers execution. We introduce Harness-R1, the first method, to our knowledge, that makes failure-conditioned, lifecycle-wide editing of an existing executable runtime a learned capability. It post-trains a dedicated harness engineer with online reinforcement learning so that its edits are optimized for the realized task success they produce, rather than proposed by a fixed editor. A separate 9B engineer converts batches of target-agent failures into validated executable patches; fresh same-batch reruns of the frozen target provide outcome rewards, so training updates only the engineer. Cold-start supervised fine-tuning initializes this editing policy, which is then trained online with group-relative policy optimization. Across WebShop, ALFWorld, and DBBench, Harness-R1 raises vanilla Qwen3.5-9B success from 44.3% to 53.6% (+9.3 percentage points).

Introduction. Large language models serve as the decision core of tool-using agents, enabling them to interpret tasks, maintain state, and pursue complex goals through multi-turn interaction with external environments (Yao et al., 2023b; Liang et al., 2024; Li et al., 2026a; Zhang et al., 2025). Unlike a single model invocation, a deployed agent continually produces trajectories containing observations, actions, environment feedback, and task outcomes. These trajectories record successful experience, but they also expose systematic failures such as tool misuse, lost state, protocol violations, repeated attempts, and failed recovery. This raises a natural question: can agents use their interaction experience to improve continually rather than remain fixed after deployment? This experience-to-improvement loop is a central concern of self-evolving agents (Gao et al., 2026; Wu et al., 2026; Yu et al., 2026). An agent system can improve at two complementary locations.

Discussion / Conclusion. As agents begin to improve other agents, harness editing becomes a form of AI improving AI. In this setting, producing edits that merely look correct is not enough: an edit intervenes directly in a running executable system, so it must be precise, verifiable, and genuinely beneficial to the agent it modifies. This is why Harness-R1 learns from the realized task outcome of each patch rather than from whether its text appears reasonable. Training a dedicated engineer beats prompting a larger model. As shown in the introduction, prompting strong but fixed frontier models to edit the harness is unreliable: they optimize for plausibility, emitting syntactically valid and reasonable-looking edits, but because they never rerun the target they cannot tell whether an edit actually raises task success, so their gains are unstable and sometimes even lower reward.