An AI can invent endless tweaks to itself, but it can only get better if it can tell which ones actually work.

This explores why an agent that can propose changes to itself still can't always tell which of those changes are real improvements, and how that limits self-improvement. The core idea, formalized in What limits how much models can improve themselves?, is that a model can improve itself only when it's better at checking solutions than at producing them. Generate a batch of attempts, keep the ones you can recognize as good, and learn from those. If checking is no easier than producing, the filter lets through as much junk as it removes, and nothing is gained. The gap grows with model size but vanishes for factual tasks. A model that doesn't know a fact can't verify it either. That's why the gap predicts which domains can bootstrap themselves and which can't.

The limit is structural, not a matter of trying harder. Can models reliably improve themselves without external feedback? lists the gap alongside diversity collapse and reward hacking as reasons pure self-improvement loops back on itself. A model grading its own work shares its own blind spots. The methods that do work all bring in something from outside: past model versions, third-party judges, user corrections, or tool feedback. Each one is a verifier whose judgment doesn't depend on the agent's current beliefs. The gap isn't closed so much as sidestepped by importing verification from elsewhere.

Agent research shows the same pattern. The Darwin Gödel Machine (Can AI systems improve themselves through trial and error?) gave up on proving that a change is better and runs each variant against a benchmark instead, keeping an archive of what works. It reached 2.5× on SWE-bench. The agent supplies the ideas, and the benchmark supplies the verdict. Most recent progress is in the fast loop that edits prompts, memory, and tools rather than weights (Do self-improving agents really split into two distinct loops?). Those edits are cheap and reversible, so a change that looked good and wasn't can be rolled back. Better generation also isn't what limits things. Producing useful harness edits is flat across model tiers, and the ability to benefit from them peaks in mid-tier models (Do stronger models always evolve harnesses better?).

Outside verifiers have their own ceiling, though. A fixed benchmark saturates and invites gaming as agents get stronger, and one proposed fix (Why do fixed benchmarks fail as agents grow stronger?) changes the criteria between epochs so the target moves faster than the agent can exploit it. Co-evolution work argues that a single agent stalls in a static context, and that peers, adaptive environments, and evolving feedback supply pressure the agent can't generate alone (Can agents evolve beyond the constraints humans engineer?). Independent verification is also one of the things a single agent loop can't organize by itself (Do single agents always hit organizational limits?).

So an agent can improve only as far as it has a judge it can't fool, and the limit shows up in the judging, not in the ideas. That judge can be a stronger checker inside the model, or a test, a peer, or a moving benchmark outside it.