When checking an AI's answer is far cheaper than producing one, does it just polish old solutions or find new ones?

This explores what changes when checking an answer is far cheaper than producing one, and whether that just polishes known solutions or lets AI find new ones. The corpus suggests the two blur together. A cheap, objective checker is what makes discovery feasible, because it lets a generate-and-test loop run long enough to pay off. AlphaEvolve is the clearest case. Automated evaluators sustained an evolutionary loop long enough to produce faster algorithms, better hardware designs and improved training methods, and the key was that cheap, objective verification closes the gap between generating candidates and knowing which are good Can machine feedback sustain discovery at test time?. Optimization is climbing toward a known target. Discovery is the same climb run long enough, with enough variety, that it lands somewhere you didn't plan for.

The Darwin Gödel Machine shows how the evaluator shapes that climb. It replaces formal proofs of improvement with plain empirical benchmarking, and it keeps an archive of agent variants rather than only the current best. The result is open-ended self-improvement, including capabilities like better code editing and context management that nobody specified up front Can AI systems improve themselves through trial and error?. The same asymmetry works at a smaller scale inside a single research task. AREX nests an outer audit loop around an inner research loop. It finds which constraints of the answer are still unresolved and sends targeted follow-up work at them, which beats simply searching longer down one path Should research agents verify answers before searching longer?. Here the cheap check is what stops early mistakes from persisting.

The catch is that all of this works where checking is possible. A 3B model reaches frontier-level scores on hard math and coding benchmarks, and the note says outright that this is bounded to verifiable tasks where reinforcement learning gets clean reward signals Can small models match frontier reasoning without massive scale?. Outside that boundary, language models generate plausible candidates but can't reliably judge their value or their own uncertainty. One line of work fixes this by pairing them with a surrogate model fitted to real experimental data Can language models reliably judge their own candidate quality?. A competing approach drops the external checker and uses the model's own token probabilities and confidence as the reward, which extends reasoning training to domains without answer keys Can model confidence alone replace external answer verification?. These two results pull in different directions on whether a model can be trusted to grade itself.

The cheap evaluator is also a dial you can turn. Verification accuracy improves independently of training, through finer score granularity, repeated evaluation and breaking criteria into parts. That reframes weak verifiers as under-scaled rather than fundamentally limited Can verification accuracy scale without training models?. Reward models that reason before scoring raise the ceiling of what can be evaluated Can reward models benefit from reasoning before scoring?. Agent-based judges that collect evidence cut judge shift from 31% to 0.27% against plain LLM judges, though a memory module that cascaded errors shows the evaluator can fail too Can agents evaluate AI outputs more reliably than language models?. Verifiers can run alongside generation at near-zero latency cost Can verifiers monitor reasoning without slowing generation down?. An inference-scaled reviewer even caught math errors that passed human review at top venues Can inference scaling help reviewers catch errors humans miss?. Each improvement widens the set of problems where discovery-by-search is affordable.

The corpus is thin on what happens when the evaluator is wrong or gameable. Every result above assumes the checker is trustworthy, and the loop can only find what the checker can recognize. So a better verifier decides which problems can be searched at all, and that matters as much as making discovery cheaper.