INQUIRING LINE

The plumbing that connects an AI to real actions is harder to edit than the AI itself — one behavior hides in a dozen places.

What makes harnesses more tangled than other types of agent code?

This explores why the 'harness' — the infrastructure code that connects a model's outputs to real actions — is harder to reason about and edit than the model or the artifacts it produces at runtime.


This explores why the harness — the infrastructure layer that wires a model's outputs to tools, prompts, and staged actions — ends up more tangled than other agent code. The starting point is that agent code isn't one thing. It splits into three layers with different failure modes: the model's trained internal capability, the system-provided harness that turns outputs into actions, and the artifacts the agent writes while running What are the three distinct layers of agent code?. The harness is the awkward middle layer, and its tangle comes from a specific structural fact about how it stores behavior.

The core reason: a single *behavior* in a harness is smeared across many *locations* in the code. Prompts, tool definitions, and pipeline stages each hold a piece of any given behavior, so there's a mismatch between how you'd describe a change ('make it stop retrying dead endpoints') and where that behavior actually lives structurally. The hardest part of changing a harness isn't writing the edit — it's finding every place the behavior is implemented. That localization problem, not code generation, is the real bottleneck Why is finding distributed behavior code so hard?. Tangle here is literally distribution: no one file is the behavior.

That framing pays off when you try to fix it. If you reorganize the repository around runtime behavior — an explicit map from behaviors to the code that implements them — even a weaker planner can locate the right spots as well as a much stronger model, with double-digit win-rate gains and fewer tokens spent hunting Can explicit behavior maps help weaker planners compete with stronger models?. In other words, the tangle isn't intrinsic to the task; it's an artifact of organizing code structurally instead of behaviorally.

There's a second kind of tangle worth naming: when the harness tries to improve itself, its 'learning' is easy to overstate. Inspecting evolved harness edits shows most of them just cache task-specific fixes the agent could have rediscovered in a single run, rather than distilling reusable strategy Do harness edits learn strategies or just memorize fixes?. And because harness evolution is itself a search loop, apparent design gains are confounded with raw search effort unless you compare against an equal-budget baseline on held-out tasks Are harness evolution gains really from better design?. So the harness is tangled both to read (behavior is distributed) and to credit (improvement blurs with memorization and search).

The deeper lesson the corpus keeps circling: harness tangle tracks how integration is done. Production teams find that protocol-mediated tool access (like MCP) introduces non-deterministic failures through ambiguous tool selection, and that replacing it with explicit direct function calls restores predictability Why do protocol-based tool integrations fail in production workflows? — with 85% of teams building custom agents rather than adopting frameworks Should coordination protocols wrap existing systems or replace them?. The through-line: harnesses get tangled when a behavior has no single home. Whether the fix is behavior-centric repo organization or deterministic direct calls, the win comes from collapsing 'where does this behavior live?' back down to one answerable place.


Sources 7 notes

What are the three distinct layers of agent code?

Long-running agentic systems decompose into model-internal capabilities (trained reasoning), system-provided harness (infrastructure connecting outputs to actions), and agent-initiated artifacts (code created during execution). Each layer fails and improves differently, and this separation clarifies where to intervene.

Why is finding distributed behavior code so hard?

The core difficulty in evolving production harnesses is not generating edits but finding every code location that implements a behavior. Harnesses distribute single behaviors across files, functions, and stages, creating a representational mismatch between behavioral requests and structural code organization.

Can explicit behavior maps help weaker planners compete with stronger models?

A behavior-to-code mapping representation improved win rates by 10–19 points while reducing planner tokens by 8–13%. Weaker planners using this mapping matched stronger models' code localization across all precision and recall metrics.

Do harness edits learn strategies or just memorize fixes?

Inspecting evolved harness edits reveals rational changes across prompt and tool layers, but most persist fixes the agent could rediscover in one rollout. Gains remain limited because edits cache shortcuts for already-solvable tasks rather than converting failures into successes.

Are harness evolution gains really from better design?

Harness evolution is itself a search loop, so its reported improvements are confounded with search effort. Only improvements beyond equal task-level search budget can be attributed to harness design, and held-out evaluation is needed to rule out task memorization.

Show all 7 sources
Why do protocol-based tool integrations fail in production workflows?

MCP integration caused non-deterministic failures through ambiguous tool selection and parameter inference. Replacing it with explicit direct function calls and single-tool-per-agent design restored determinism. A 306-practitioner survey confirms 85% of production teams build custom agents, forgoing frameworks.

Should coordination protocols wrap existing systems or replace them?

Research shows that agent coordination standards achieve adoption by composing existing protocols like MCP and DIDComm under a shared substrate, rather than competing to replace them. Bridging lets value accrue incrementally without forcing ecosystem-wide rewrites.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.

Research prompt for your LLMexpand ↓

Copy into ChatGPT or Claude to take this line of inquiry further — it asks the model to find newer work and re-test which earlier constraints still hold.

You are a systems analyst studying agentic AI. Question, still open: what makes harnesses more tangled than other types of agent code?

What a curated library found — and when (dated claims, not current truth). These span roughly 2025-06 to 2026-06:
- Agent code splits into three layers — trained model capability, the system-provided harness, and runtime artifacts — and the harness is the awkward middle (~2026).
- A single behavior is smeared across many locations: prompts, tool definitions, and pipeline stages each hold a piece, so localization — finding every place a behavior lives — not code generation is the real bottleneck (~2026).
- Reorganizing a repo around runtime behavior lets a weaker planner locate the right spots as well as a much stronger model, with double-digit win-rate gains and fewer tokens (~2026).
- Evolved harness edits mostly cache task-specific fixes rather than distill reusable strategy; apparent gains blur with raw search effort absent an equal-budget baseline (~2026).
- Protocol-mediated tool access (MCP) adds non-deterministic failures via ambiguous tool selection; direct function calls restore predictability, and 85% of teams build custom agents (~2025).

Anchor papers (verify; mind their dates): Code as Agent Harness (arXiv:2605.18747, 2026); Harness Updating Is Not Harness Benefit (arXiv:2605.30621, 2026); A Practical Guide for Production-Grade Agentic AI (arXiv:2512.08769, 2025); From Model Scaling to System Scaling (arXiv:2605.26112, 2026).

Your task: (1) RE-TEST EACH CONSTRAINT — for every finding, judge whether newer models, training, tooling, orchestration (memory, caching, multi-agent), or evaluation has RELAXED or OVERTURNED it; separate the durable localization question from perishable limits, cite what resolved each, and say where a constraint still holds. (2) Surface the sharpest DISAGREEMENT in the last ~6 months — especially MCP/protocol layers vs. direct-call determinism, and whether behavior-centric reorg beats it. (3) Propose 2 research questions assuming the regime has moved.

Cite arXiv IDs; flag anything you cannot ground in a real paper.