SYNTHESIS NOTE
TopicsAgent Harnessthis note

Can explicit behavior maps help weaker planners compete with stronger models?

Explores whether organizing harness repositories around runtime behavior—rather than relying on model inference—can narrow the capability gap between weaker and stronger planning models, and whether this reduces computational overhead.

Synthesis note · 2026-07-17 · sourced from Agent Harness
Where does agent reliability actually come from? How do you navigate synthesis across fragmented research topics?

If Why is finding distributed behavior code so hard?, the fix is to build the missing map explicitly rather than make an agent recover it every time. Harness Handbook is a behavior-centric representation: it links behavioral descriptions to the distributed source that implements them, constructs that mapping directly from source, and supports editing through behavior-guided progressive disclosure plus automatic resynchronization when the code changes.

The results argue that the mapping — not raw model strength — was doing the work. With the handbook, overall win rates rise 10.0 and 18.9 points on Codex and Terminus-2 while planner token use falls 12.7% and 8.6%: better plans for less compute, because the planner no longer burns tokens rediscovering where behavior lives. The sharper evidence is substitutive: with the handbook a weaker planner matches the implementation-site localization of substantially stronger models, improving every file- and symbol-level Recall/Precision/F1 comparison against two independent reference plans, and the gains persist across request types and difficulty levels.

This is a concrete instance of a recurring pattern in the harness literature — capability relocating out of the model and into the scaffolding around it. Like Can externalized bookkeeping let smaller search agents beat larger ones?, the move is to externalize a structure (here, the behavior→code map) so the model spends its budget on judgment rather than bookkeeping. It also complements What makes agent-authored code worth persisting and sharing?: a durable, resynchronized representation is exactly the kind of persistent artifact that compounds.

Inquiring lines that read this note 15

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Do harness improvements transfer across model scales or memorize shortcuts? Can language model RL training avoid reward hacking and misalignment? How do prompt structure and constraints affect model instruction reliability? Does externalizing cognitive work and state improve agent reliability? Does decoupling planning from execution improve multi-step reasoning accuracy? Can single-axis benchmarks accurately predict agent deployment success? How should systems govern persistent agent-generated code in shared infrastructure?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 61 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

reorganizing a harness repository around runtime behavior lets a weaker planner match a stronger model's code localization