Swap the scaffolding around a coding AI and its bill swings wildly — but how well it actually codes barely budges.

This explores what changes when you swap the harness around a coding agent, meaning the system prompts, tool definitions and control loop that wrap the model. The short answer from the corpus is that cost moves a lot and accuracy barely does. In a test of three production harnesses and five models on SWE-bench Verified, switching harnesses shifted pass rates by no more than ordinary rerun variation, but it changed token cost by up to threefold Does switching between coding harnesses change model accuracy?. The cost gap has a simple cause. Each harness carries a fixed overhead in its system prompt and tool schemas, and that overhead is paid again on every step. A harness that is a little wordier and takes a few more steps ends up much more expensive.

The less obvious point is that the two measurements are not equally reliable. Cost is mechanical: the same harness on the same task uses roughly the same number of tokens, so differences show up clearly. Accuracy is noisy. Rerunning the same harness flips task outcomes about as often as switching to a different harness does. On a typical 45-task comparison, a real 13-point accuracy gap is caught only about half the time Do harness comparisons on SWE-bench Verified measure real differences?. So when someone says harness A beats harness B on accuracy, the first question to ask is whether they reran the same harness to see how much the score moves by chance alone.

This changes how harness work looks. If cost is the lever that reliably moves, then the most measurable progress is in efficiency. That matches what large-scale automated harness search found: four mechanisms (how actions are run, how context is compacted, how observations are handled, and handing off reading to a sub-step) cut token traffic by 44.7–49.0% while performance stayed comparable Can agent harnesses be automatically optimized across many environments?. The same tension also runs the other way. When a harness change does seem to raise accuracy, it may be because it spent more compute. Gains from automatic harness evolution count only if they beat a plain search baseline given the same feedback and inference budget How should we measure gains from automatic harness evolution?.

This doesn't mean harnesses never matter for accuracy. Harnesses that are evolved or revised for specific benchmarks report large gains with the model left unchanged: about 17 points on average for DarwinX Can frozen models improve by evolving their harnesses?, and harness fixes beating extra fine-tuning in RSIGym Do harness fixes or heavier training drive frontier model gains?. The way to reconcile the two findings is the difference between swapping and tailoring. Choosing between mature, general-purpose harnesses mostly changes your bill. Building a harness for a specific task can change your score, but only a rerun baseline and a matched compute budget will show whether that gain is real.