Could an AI evaluator that keeps raising its own bar force models to genuinely improve instead of gaming a fixed score?

This explores whether an evaluator that keeps raising its own bar can act like a curriculum for agents — and the corpus is unusually direct on the question. The core insight comes from work on why fixed benchmarks fail: static evaluation criteria saturate and invite reward hacking as agents get stronger, because once the target stops moving, the optimizer's easiest win is to exploit the scorer rather than get better at the task Why do fixed benchmarks fail as agents get stronger?. The proposed fix is precisely a non-stationary objective — the Red Queen Gödel Machine splits search into epochs with stable criteria *within* an epoch but evolves the utility function *across* epoch boundaries, keeping the target just ahead of the optimizer. That epoch structure is what makes it curriculum-like rather than chaotic: each stage is gradable, but clearing it unlocks a harder standard.

The companion idea is that the evaluator shouldn't just get stricter on a fixed axis — it should *learn alongside the agent*. Co-evolving the evaluator with the agent extends recursive self-improvement to tasks that have no static verifier at all, like writing and proof generation Can evaluators improve alongside the agents they score?. Notably, co-evolved systems match fixed-evaluator performance while spending fewer tokens, which hints that a shared, adapting standard transmits information to the agent more efficiently than a frozen rubric — the evaluator is teaching, not just gatekeeping.

There's a deeper reason a progressive evaluator matters, and it's about the ceiling on learning. Agents trained on static expert demonstrations are capped by the curator's imagination — they never interact with an environment, so they can't learn from their own failures or exceed what was demonstrated Can agents learn beyond what their training data shows?. A stricter-over-time evaluator is one way to break that cap without hand-authoring harder examples: the raising bar generates the next lesson automatically from the agent's current frontier. A related move is to make the *curator itself* trainable — SkillOS separates a learning curator from a frozen executor, and the curator learns to evolve a skill library toward strategic meta-skills rather than verbose clutter Can a separate trained curator improve skill libraries better than frozen agents?. That's the same shape from the supply side: a separate component that keeps upgrading what "good" means.

Two cautions keep the idea honest. First, a stricter evaluator only teaches if it's scoring the right thing — evaluation has to expand from final answers to whole trajectories, judging process quality, recoverability, and coordination, or the curriculum just rewards better final-answer guessing How should we evaluate agent behavior beyond final answers?. Rewarding the *reasoning* directly, as with verifiable meta-reasoning rewards for planning and reflection, is one way to make the rising bar pull on skill rather than luck Can RL agents learn to reason better, not just succeed?. Second, a harsher evaluator is only useful if it's *reliable* — agentic judges that collect their own evidence cut judge error by two orders of magnitude over LLM-as-a-judge, but even they cascade errors through a faulty memory module Can agents evaluate AI outputs more reliably than language models?. A curriculum built on a noisy grader teaches noise.

The thing you might not have expected: the strongest version of this isn't "crank up the difficulty knob." It's that the evaluator and the agent are locked in a Red Queen race where *both* keep moving, and the win condition is staying just ahead — strict enough to prevent the agent from coasting into reward hacking, gradable enough within each epoch that progress is still possible.