When an AI agent's setup keeps failing, does tracing what it does while running show you which code to fix?

This explores whether organizing a harness by what it does at runtime, instead of by how its code is laid out, helps you find where the harness is going wrong. The corpus says yes. It also suggests that finding the problem is the harder half of fixing it.

The difficulty comes from a mismatch. A complaint about a harness is behavioral, such as "it handles failed tool calls badly." But harnesses spread one behavior across many files, functions and stages, so no single place holds it. Why is finding distributed behavior code so hard? argues this is the real bottleneck in evolving production harnesses. Writing the edit is easy once you know every location that implements the behavior, and locating them all is where things break down.

The most direct evidence is a behavior-to-code map, which lists each runtime behavior next to the code that implements it. Can explicit behavior maps help weaker planners compete with stronger models? found that this raised win rates by 10–19 points and cut planner tokens by 8–13%. Weaker planners using the map matched stronger models' code localization on every precision and recall metric. So most of the gap between weak and strong planners was a lookup problem, and a good representation removed it without a bigger model.

Three other findings show why a map is needed. Do harness edits learn reusable strategies or memorize task fixes? shows that most evolved edits just record fixes an agent could have rediscovered in one rollout, so they patch symptoms instead of the underlying deficiency. Can language models build and maintain their own agent harnesses? shows that harness quality can't be read off downstream task scores, so a score tells you something is wrong but not where. Do stronger models always evolve harnesses better? shows that weak models often fail to invoke a harness at all, and strong models struggle to follow it faithfully. The notes don't test whether a behavior map makes edits less memorized. That link is a reasonable guess, not a result.

Behaviors also look like the natural unit for this work. Can agent harnesses be automatically optimized across many environments? found four mechanisms (action execution, context compaction, observation handling, delegated reading) that cut token traffic by 44.7–49.0%. Can harness modules improve separately from benchmark data? improves modules separately, pooling evidence across tasks before touching a component. Both treat the harness as a set of separable behaviors. The raw material for a map may already exist, since Can a routing harness generate its own training data automatically? notes that a deployed routing harness already records execution trajectories and outcomes.

There is a limit on how far the map carries. Should safety harnesses be customized for each deployment? shows that a harness strict enough for one model over-blocks another, so a deficiency depends on the model and the deployment as well as the code. A map would likely need rebuilding for each one. The corpus proves the point for code localization in one setting. It has nothing yet on using behavior maps to diagnose safety or reasoning failures, or on whether they hold up across models.