SYNTHESIS NOTE
Topics›Agent Harness›this note

Can explicit behavior maps help weaker planners compete with stronger models?

Explores whether organizing harness repositories around runtime behavior—rather than relying on model inference—can narrow the capability gap between weaker and stronger planning models, and whether this reduces computational overhead.

Synthesis note · 2026-07-17 · sourced from Agent Harness

If Why is finding distributed behavior code so hard?, the fix is to build the missing map explicitly rather than make an agent recover it every time. Harness Handbook is a behavior-centric representation: it links behavioral descriptions to the distributed source that implements them, constructs that mapping directly from source, and supports editing through behavior-guided progressive disclosure plus automatic resynchronization when the code changes.

The results argue that the mapping — not raw model strength — was doing the work. With the handbook, overall win rates rise 10.0 and 18.9 points on Codex and Terminus-2 while planner token use falls 12.7% and 8.6%: better plans for less compute, because the planner no longer burns tokens rediscovering where behavior lives. The sharper evidence is substitutive: with the handbook a weaker planner matches the implementation-site localization of substantially stronger models, improving every file- and symbol-level Recall/Precision/F1 comparison against two independent reference plans, and the gains persist across request types and difficulty levels.

This is a concrete instance of a recurring pattern in the harness literature — capability relocating out of the model and into the scaffolding around it. Like Can externalized bookkeeping let smaller search agents beat larger ones?, the move is to externalize a structure (here, the behavior→code map) so the model spends its budget on judgment rather than bookkeeping. It also complements What happens to code that agents create and then share?: a durable, resynchronized representation is exactly the kind of persistent artifact that compounds.

Inquiring lines that read this note 61

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Can harness optimization rival model scaling in effectiveness? How can reward models fail to detect reward-seeking disguised as alignment? Which agent-created code should persist in infrastructure versus remain task-specific? Why does separating planning from execution enhance reasoning performance? Do standard benchmarks reveal AI capability in real-world contexts? Does evolutionary optimization at test time improve model performance effectively? How do different post-training structures affect learning outcomes? Can AI systems autonomously pursue unstated objectives or circumvent design constraints? Do benchmark score improvements actually reflect genuine AI capability gains? How does model awareness of deployment context affect strategic behavior? Does AI-assisted coding actually improve developer productivity and code quality? Can AI agents learn to improve performance from experience? Do single-axis benchmarks adequately measure deployment-ready agent capability? How should agents balance abstract planning with concrete environmental grounding? How bounded is recursive self-improvement in current AI systems? How should control and autonomy be allocated in human-AI collaboration? How do benchmark scores relate to real-world task completion? Does expanding agent autonomy undermine human oversight capacity? How does transparency of agent reasoning affect effective oversight? How should agent memory systems balance granularity, management, and reuse? How should organizational structure guide multi-agent system design? When does routing outperform scaling as a performance solution?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 80 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

reorganizing a harness repository around runtime behavior lets a weaker planner match a stronger model's code localization