Which coding harness components matter most in different conditions?

The paper's central claim is that what a coding-harness component buys depends on the setting, so it cannot be judged as a fixed feature. The authors hold the execution loop constant and vary three components (planning, action space, context management) across four models on SWE-Bench Verified and Terminal-Bench 2.1, in 176 matched settings that include five context-management strategies and four context-window budgets. Their findings are conditional. Context management "becomes increasingly valuable as the context-window budget tightens." Planning "shifts from an accuracy scaffold for weaker models to a cost saver for stronger models." The conclusion adds that predefined tools raise success for models with weak bash control, while bash-only gives higher success at lower cost for bash-capable models.

The paper frames this as a gap in prior work, which "typically evaluates harnesses as monolithic systems, leaving the effectiveness of individual components unclear." Its components each intervene differently: a planning scaffold maintains task structure, an action interface turns model intentions into executable operations, and a context policy decides what history survives "under a finite window." The context result comes with a stated mechanism, since most of the benefit comes "from preventing context-overflow failures." Among strategies, staging rule-based elision before selective LLM summarization gives the lowest aggregate cost at broadly similar success rates, while making elided content recoverable "adds machinery that models rarely use and yields no accuracy gain."

This sharpens the neighbors in the library. Can an external manager handle context for frozen agents? indexes the right amount of context to the agent's competence; this paper indexes context management's value to budget, and planning and action space to model strength. That is a different axis, and the excerpt does not say whether the best context strategy also shifts with the model. Where does agent reliability actually come from? treats externalization as the source of reliability. The recoverable-elision result qualifies that: keeping dropped content retrievable is the nearest analogue to externalized memory in this design space, and here models rarely used it. Can explicit behavior maps help weaker planners compete with stronger models? is a second case where a harness-side change helps a weaker planner most. The study also gives controlled component-level evidence inside the system-provided layer of What are the three distinct layers of agent code?.

The excerpt does not name the four models, report effect sizes or budget values, or give any uncertainty. It states the planning cost for stronger models only as "small decreases in success rate," and it does not say how "weaker" and "stronger" were determined beyond the models themselves. The results come from one lightweight harness and two coding benchmarks, so nothing here shows they carry over to other loops, other harnesses or non-coding tasks. What follows at this strength is a design habit: choose harness components against the model and the window budget in use, because a component that lifts accuracy in one setting can be pure overhead in another.