Self-improving AI tooling mostly memorizes fixes for tasks it could already do — the genuinely hard failures go unaddressed.

This explores what stays broken even after harnesses are made to improve themselves. The short version: the corpus suggests harness evolution has been surprisingly good at the wrong thing. When researchers actually inspect the edits an evolving harness makes, most of them just cache fixes for tasks the agent could already have solved in a single try — the changes memorize task-specific patches rather than distilling reusable strategy Do harness edits learn strategies or just memorize fixes?. So the headline gains are partly an illusion of progress: the hard failures (turning things the agent *can't* do into things it can) remain largely untouched.

A second unsolved problem is measurement itself. Harness evolution is a search loop, so any improvement it reports is tangled up with raw search effort. Unless you compare against a matched compute budget spent on plain test-time search, you can't actually credit the *design* — and without held-out evaluation you can't rule out that the harness simply memorized the benchmark Are harness evolution gains really from better design?. This connects to a broader decoupling insight: only when you split evaluation into separate benchmark, harness, and environment components do failure modes like reward-hacking become visible at all, because scalar scores hide them How can separating evaluation components make reward-hacking visible?.

Underneath the memorization problem sits a stubborn technical bottleneck: finding where a behavior actually lives in the code. Real harnesses smear a single behavior across many files, functions, and stages, so the genuinely hard part isn't *generating* an edit — it's *localizing* every place that needs to change Why is finding distributed behavior code so hard?. One promising response is reorganizing the harness around runtime behavior rather than file structure, which lets even a weaker planner match a stronger model at locating the right code Can explicit behavior maps help weaker planners compete with stronger models? — but notice this fixes navigation, not the deeper capability gap.

And there's a ceiling that harness tinkering can't push through. The capacity to *produce* useful harness edits is roughly flat across model sizes, but the capacity to *benefit* from them peaks in the middle — weak models never invoke the harness, strong ones struggle to follow their own instructions faithfully Do stronger models always evolve harnesses better?. That inverted-U echoes a more fundamental result: pure self-improvement is structurally circular. It stalls on the generation–verification gap, diversity collapse, and reward hacking, and every method that reliably works is quietly smuggling in an *external* anchor — a prior model version, a third-party judge, a user correction, or real tool feedback Can models reliably improve themselves without external feedback?.

So the thing you didn't know you wanted to know: the systems that *do* keep improving aren't the ones with the cleverest self-editing loops — they're the ones that convert failure into an external signal. A pivot-or-refine executor that routes every experiment failure through a decision process keeps making progress precisely because failure becomes information rather than a dead end Can experiment failures drive progress instead of stopping it?, and the Darwin Gödel Machine's open-ended gains come from empirical benchmarking against an archive of past variants, not from formal self-reasoning Can AI systems improve themselves through trial and error?. The persistent failures harness evolution hasn't solved — genuine capability expansion, honest measurement, behavior localization, and escaping self-referential circularity — are exactly the ones that require reaching outside the loop.