Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable

The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, execution environments, and application requirements change, the harness must be continually modified to add capabilities or adapt existing behaviors. Before a human developer or coding agent can make such a change, they must identify all code locations that implement the target behavior. This is difficult because production harnesses are often large, tightly coupled, and behaviorally distributed across files, functions, execution stages, and state transitions, whereas modification requests describe what the system should do and repositories are organized by files, functions, and modules. Existing approaches to code search, repository indexing, and long-context processing make code easier to inspect, but they still leave developers and coding agents to recover this mapping themselves. Behavior localization is therefore a central bottleneck in harness evolution.

Introduction. Recent advances in large language models (LLMs) have accelerated the development of agentic systems [1–3]. These systems extend model capabilities by interacting with tools, APIs, and complex execution environments [4–7]. Such interactions are not governed by the foundation model alone. They are coordinated by the harness, which constructs prompts, manages state, invokes tools, and controls execution across system components. The harness therefore determines how model capabilities are translated into system behavior. As models, APIs, execution environments, and application requirements evolve, the harness must be adapted accordingly. Harness evolution has consequently become a recurring engineering challenge in modern agent development [8–10]. In practice, harness evolution requires changes that add capabilities, adapt existing behaviors, or refine execution workflows [11, 12]. Traditionally, human developers make these changes by reading the repository, tracing the relevant behavior, and editing its implementation.

Discussion / Conclusion. This work identifies behavior localization as a central challenge in evolving production-scale agent harnesses and introduces Harness Handbook, a behavior-centric representation that links behavioral descriptions to their distributed source implementations. Harness Handbook reorganizes repository knowledge around runtime behavior, constructs this representation directly from source, and supports modification through behavior-guided progressive disclosure and automatic resynchronization. Our experiments demonstrate the value of this representation along three dimensions. First, handbook guidance improves plan quality while lowering planning cost: on Codex and Terminus-2, overall win rates rise by 10.0 and 18.9 percentage points, while planner token use falls by 12.7% and 8.6%, respectively. Second, with the handbook a weaker planner matches the implementation-site localization of substantially stronger models, improving all 24 file- and symbol-level Recall, Precision, and F1 comparisons against two independent reference plans. Third, these gains persist across request types and localization difficulty levels.