If an AI grader keeps learning alongside the AI it grades, does that make its scores right, or just current?

This explores whether letting the scorer learn and change alongside the agent it scores can replace a trustworthy benchmark on tasks like creative writing or open-ended proofs, where no answer key exists. The corpus says co-evolution removes one bottleneck, but it doesn't yet show that the evaluator is right. It only shows that the evaluator keeps up.

The positive result is real. The Red Queen Gödel Machine makes evaluation part of the improvement loop, so agents get better at writing and proof generation without a fixed verifier (Can evaluators improve alongside the agents they score?). Co-evolved systems match fixed-evaluator performance while using fewer tokens. The catch is what 'match' means: the comparison is against a fixed evaluator. That shows the co-evolved scorer is no worse as a training signal. It doesn't show the scorer is correct on tasks where no fixed evaluator can be trusted. The broader survey explains why the approach is attractive: a single agent improving against a static judge stalls, and co-evolving components supply adaptive pressure (Can agents evolve beyond the constraints humans engineer?).

The self-improvement systems with the cleanest evidence lean on benchmarks. The Darwin Gödel Machine swaps formal proofs for empirical benchmarking and gets 2.5× on SWE-bench (Can AI systems improve themselves through trial and error?). AIDE2's evolved agent is checked on four held-out benchmarks, including physics-based weather forecasting outside its selection distribution, which is how the authors rule out overfitting (Do AIDE2's improvements transfer to unseen tasks?, Does automated evolution match human-built agent performance?). Those checks are possible only because the tasks are benchmarkable. On hard-to-benchmark tasks that outside check is missing, and co-evolution doesn't supply it.

The corpus offers other ways to make judging trustworthy, and they work by improving the evaluator directly. One is to break subjective quality into verifiable sub-criteria. Checklist rewards do this and reduce overfitting to superficial artifacts (Can breaking down instructions into checklists improve AI reward signals?). Another is to let the judge think before scoring, which raises the ceiling of reward models beyond outcome-only scoring (Can reward models benefit from reasoning before scoring?). A third is to have an agent gather evidence before judging. That cut judge shift to 0.27%, against 31% for an LLM judge on complex tasks. But its memory module cascaded errors (Can agents evaluate AI outputs more reliably than language models?). That warning applies to any evaluator that accumulates state. A co-evolving judge that learns from its own past verdicts could compound its mistakes unless something isolates them. A related idea has models learn to score their own work during training, at zero inference cost (Can models learn to evaluate their own work during training?). That shifts where the judge lives, but it doesn't settle who checks the judge.

So co-evolution keeps the evaluator from becoming a fixed ceiling on progress, and that is a different problem from verification. The corpus has no test of a co-evolved evaluator against human judgment on an unbenchmarkable task. The most defensible reading is that co-evolution is one layer in a stack with decomposed criteria, reasoning judges and evidence-gathering, and the layer that would give ground truth is still open.