Does switching between coding harnesses change model accuracy?

The paper runs five models through three production coding-agent harnesses, Claude Code, mini-SWE-agent, and OpenCode, on SWE-bench Verified, and reruns the same configurations to measure how much a score moves from noise alone. On 447 tasks with the two self-served models, "Claude Code and mini-SWE-agent, the heaviest and the lightest harness, are equivalent within five points." On a 45-task hard subset across five models, "swapping the harness flips as many tasks as rerunning the same harness, 13% in both cases, and the tasks a harness wins in one run are not the tasks it wins in the next." The one effect that clears the noise is a loss: OpenCode trails by up to 9 points on the large pool, tied to output-cap cutoffs rather than anything about reasoning quality. What does move, reliably, is the bill: "with the same model, the same tasks, and one price list, cost per task differs by up to 3× across harnesses."

The mechanism the authors give is specific: "the gap is set at the first call, by the preamble of system prompt and tool schemas each harness sends with every step, and scaled by the number of steps; per-step growth and per-call tool output differ far less." The provider's price for cached input "scales the bill and does not reorder it" — it changes the size of the gap, not which harness is cheaper. In other words, the fixed overhead a harness pays before the model does any work, multiplied by however many steps the task takes, explains most of the cost spread; what happens inside each step contributes far less.

This cuts against where the library's other harness-value findings locate the payoff. Can agent harnesses be automatically optimized across many environments? reports a search-discovered harness stack that cut token traffic 44.7–49.0% "at comparable performance" — consistent with this paper's picture that harness engineering buys efficiency, not accuracy, though that note's savings come from mechanism selection inside one harness rather than from swapping whole production harnesses. Which coding harness components matter most in different conditions? finds individual components (planning, context management) help accuracy under specific conditions (tight windows, weak models); this paper's whole-harness, rerun-calibrated comparison suggests those conditional accuracy gains, if real, are smaller than the run-to-run noise its own data shows for production harnesses — a reason to read single-run component results with the same caution this paper applies to single-run harness comparisons.

The excerpt's noise floor is calibrated only for these three harnesses, five models, and one benchmark; it doesn't say whether smaller or larger pass-rate gaps on other benchmarks would clear the same bar. The closed anchor model (Claude Opus 5) ran inside Claude Code only, so the paper cannot say whether a frontier closed model shows the same harness-indifference. The defensible reading is narrower than "harnesses don't matter": on this benchmark and this model ladder, harness choice is a cost decision with an accuracy effect too small to separate from noise, and claims of harness-driven accuracy gains elsewhere need the same rerun control to be believed.