Does a smarter AI always get more out of its tools and memory setup? Turns out, the middle of the pack wins.
How do model tier and harness quality interact in agent self-improvement?
This explores whether a stronger model automatically gets more out of a self-improving agent harness (the prompts, memory, tools and control code wrapped around it), or whether model strength and harness quality interact in less obvious ways.
This explores whether a stronger model automatically gets more out of a self-improving agent harness (the prompts, memory, tools and control code around it), or whether the two interact in less obvious ways. The corpus says they don't simply add up. One study found that models of every tier are about equally good at writing useful harness edits, but the ability to benefit from those edits follows an inverted U that peaks in mid-tier models Do stronger models always evolve harnesses better?. Weak models fail to invoke the harness at all. The strongest models struggle to follow its instructions faithfully.
The harness is worth studying because most of the recent progress in self-improvement is happening there. Agents split into a slow loop that updates weights and a fast loop that updates prompts, memory and tools. Progress concentrates in the fast loop because scaffold changes are cheaper and reversible Do self-improving agents really split into two distinct loops?. The harness also does real work. Reliable agents move memory, skills and interaction protocols out of the model and into the harness, so the model doesn't have to solve the same problems repeatedly Where does agent reliability actually come from?. On the results side, optimizing the execution system around frozen weights lifted several models on Terminal-Bench, and the same runbook carried over to newer models unchanged Can execution harnesses lift model performance without retuning weights?. Automatically optimizing harnesses across many environments found four mechanisms that cut token traffic by 45-49% at comparable performance Can agent harnesses be automatically optimized across many environments?. That work describes harness gains as orthogonal to model gains.
The tier catch is that a harness is not a portable asset. Models differ sharply in how well they build harnesses, and they struggle to keep useful intermediate updates during evolution. A harness's performance also shifts dramatically when a different model executes it, so its quality can't be read off downstream task scores Can language models build and maintain their own agent harnesses?. Combined with the inverted U, this suggests a harness is a contract between designer and executor. A model too weak can't hold up its end of it, and a model too strong may drift off the script. The transfer result above shows portability is possible, but the corpus treats it as something to test rather than assume. Harness gains can survive a model swap without being equally large for every model.
Why any of this matters for self-improvement comes down to external anchors. Pure self-improvement stalls because models can't verify their own work reliably, their outputs lose diversity, and they hack their own rewards. The methods that work bring in something outside the model, such as past versions, third-party judges, user corrections or tool feedback Can models reliably improve themselves without external feedback?. The harness is where those anchors live. It also keeps the model in the loop. On long optimization tasks, repeated benchmark-edit-incorporate cycles predicted success better than initial quality, and most models quit early or wasted their budget What predicts success in ultra-long-horizon agent tasks?. A model's tier sets how well it uses that loop, and the harness decides whether the loop keeps running.
The two loops can also feed each other. A deployed routing harness records trajectories, difficulty estimates and outcomes that can be turned into training data, so the fast loop ends up supplying the slow one Can a routing harness generate its own training data automatically?. The corpus has only one study that varies tier and harness benefit head to head, so the inverted U rests on a single result. The other notes explain why the interaction is plausible, but none of them confirm it.