Making an AI smarter and changing what scores it look similar — but they break in completely opposite ways.

This explores the difference between two moves in self-improving AI systems: evolving the *optimizer* (the agent, its harness, its search strategy) versus evolving the *benchmark* (the evaluator, the objective, the scoring function). The distinction matters because they fail in opposite ways. When you only evolve the optimizer, the target stays still — and a still target eventually gets gamed. Why do fixed benchmarks fail as agents get stronger? makes this concrete: fixed benchmarks saturate and invite reward hacking as agents get stronger, because the agent's whole job is to find the shortest path to a high score, and a frozen scorer eventually reveals that path. Evolving the benchmark is a way of keeping the target ahead of the optimizer instead of underneath it.

The cleanest illustration of the split is the Red Queen setup in Can evaluators improve alongside the agents they score?, where evaluation itself becomes part of the improvement loop. This unlocks something a fixed-evaluator system can't do: self-improvement on tasks that have no static grader at all — creative writing, proof generation — where 'better' can't be pinned to a number in advance. Notice what's changed. Evolving the optimizer asks *how do I solve this better?* Evolving the benchmark asks *what should count as solved?* Can agents evolve their own objectives during search? pushes this all the way: an outer loop proposes new objectives and compiles them into scoring code the inner loop uses immediately, so goal design stops being a fixed input and becomes part of the discovery. That's a categorically different kind of learning than getting faster at hitting a goal someone handed you.

But here's the twist the corpus insists on: evolving the benchmark introduces a risk that evolving the optimizer doesn't. If the system writes its own test, who checks the test? Can models reliably improve themselves without external feedback? argues that pure self-improvement is structurally circular — it stalls on the gap between generating and verifying, and every method that actually works smuggles in an external anchor: a past model version, a third-party judge, a user correction, a tool result. So evolving the evaluator is powerful precisely because it escapes benchmark saturation, and dangerous precisely because it can collapse into a system grading its own homework. The Red Queen design handles this with a structural trick — splitting search into epochs where the criterion is stable *within* an epoch but evolves *across* boundaries, so the optimizer never gets to co-opt the judge it's currently being scored by.

There's also a measurement problem that only appears once you start evolving machinery instead of solutions. Are harness evolution gains really from better design? points out that harness evolution is itself a search loop, so its 'improvements' are confounded with raw search effort — you can't credit the design unless you match the compute budget and hold out fresh tasks. And Do harness edits learn strategies or just memorize fixes? finds that most evolved edits just cache fixes for already-solvable tasks rather than converting failures into successes. This is the deep asymmetry: a better optimizer produces gains you can read off a benchmark, but a changed benchmark changes what 'gain' even means — which is exactly why Do automated benchmarks hide what frontier AI systems can really do? argues that auto-gradable benchmarks both overstate and understate real capability. Evolving the optimizer is climbing the hill faster. Evolving the benchmark is admitting you may have been measuring the wrong hill.