When an AI rewrites its own code, how do you know it truly improved instead of just gaming the test?

This explores how you can tell that an agent rewritten by a person or by itself is really better, rather than just scoring higher on a test it was tuned for. The corpus points to several layers of validation, and none of them is a single number.

The cleanest answer is a hidden test. AIDE2 proposed changes to its own code, benchmarked the modified versions on AI R&D tasks, and kept only the rewrites that scored best on evaluations the proposing agent couldn't see. It made seven successive improvements in eight days, and each accepted rewrite became the agent that proposed the next one Can an AI agent reliably improve itself through hidden evaluation?. An agent can't tune itself to a test it can't see, so a win on that test means something.

A score still isn't enough, because a correct result can come from a bad process. Agents that skip a required verification step can still land on a verdict that matches the ground truth, so outcome-only monitoring can't tell compliance from corner-cutting Can a correct outcome hide protocol violations in multi-agent systems?. Identical success rates can also hide big differences in efficiency, reliability and deployment readiness, which is why one line of work argues for measuring the whole trajectory (memory hygiene, verification cost) instead of task success alone How should we measure agent system performance beyond task success?. BenchShield takes this furthest. Operators issue a claim that the agent completed the task validly, backed by recorded infrastructure evidence that it followed the intended path, instead of trusting a terminal score Can infrastructure evidence replace terminal scores in benchmark validation?. So a rewrite that seems better may just have found a shortcut.

Then there's the question of who does the judging. An agentic judge that collects its own evidence shifted only 0.27% on complex tasks, against 31% for a plain LLM-as-a-Judge. Its memory module cascaded errors, though, so the judge needs error isolation too Can agents evaluate AI outputs more reliably than language models?. When the rewrite is a code change, you can sometimes skip running it. Structured reasoning templates reached 93% accuracy at deciding whether two patches are equivalent, which is reliable enough to serve as a training reward for certain task classes Can structured reasoning replace code execution for RL rewards?.

"Better" also depends on which model runs the rewrite. Models of every tier are about equally good at producing useful harness edits, but the benefit from those edits peaks in mid-tier models. Weak models fail to invoke the harness, and strong models struggle to follow its instructions faithfully Do stronger models always evolve harnesses better?. A rewrite validated on one model may not transfer to another, so test it on the model you'll deploy. Scaffold rewrites are cheap and reversible, unlike weight updates, which is partly why self-improvement work concentrates there Do self-improving agents really split into two distinct loops?. If the checks above fail, you can roll back.

These notes suggest a rewrite counts as better when four things hold: it wins on a test the rewriter couldn't see, the evidence shows it did the work properly, the judge is trustworthy, and it holds up on your own model.