Can an AI get better just by grading its own work, or does it always need something outside to check against?
Can agents improve reliably without an external standard?
This explores whether an agent can get better using only itself as the judge, with no outside yardstick, or whether some anchor is always hiding in the loop.
This explores whether an agent can get better using only itself as the judge, with no outside yardstick, or whether some anchor is always hiding in the loop. The corpus says pure self-judgment doesn't hold up. The agent can't check its work as well as it can produce it, its outputs grow more uniform, and it learns to game whatever score it grades itself with. The methods that do improve reliably each smuggle in something external: past versions of the model, a third-party judge, user corrections, or tool feedback Can models reliably improve themselves without external feedback?.
The anchor can be quieter than a benchmark. Agents trained only on expert demonstrations never interact with an environment, so they can't learn from their own failures. Their ceiling is whatever the dataset curators imagined Can agents learn beyond what their training data shows?. The way out is letting the world push back. Apodex scales executable environments so that state, recovery and delivery become trainable behaviors Can agents learn to work reliably through environment and coordination scaling?. On ultra-long optimization tasks, the best predictor of success was how many benchmark-edit-incorporate cycles a model kept running, not how good its first attempt was. Most models quit early or burned budget without using the feedback What predicts success in ultra-long-horizon agent tasks?. So the real question is less whether an outside standard exists than whether the agent keeps listening to it.
The standard also doesn't have to be fixed, and fixed ones may be the worst option. Static benchmarks saturate and invite gaming as agents get stronger Why do fixed benchmarks fail as agents grow stronger?. Even a solid external benchmark can measure the wrong thing: agents clear abstract contests but fail long professional workflows, because the field optimized what it measured Why do agent benchmarks not predict real economic value?. Two designs treat the standard as something that moves. The Red Queen Gödel Machine keeps criteria fixed within an epoch, so progress is still checkable. It changes them between epochs, moving the target faster than the agent can exploit it Why do fixed benchmarks fail as agents grow stronger?. It can also co-evolve the evaluator with the agent. That let agents improve at writing and proof generation with no static verifier, matching fixed-evaluator results with fewer tokens Can evaluators improve alongside the agents they score?. A broader survey describes this as a progression: first peers that adapt, then environments and feedback that adapt, and finally the evolution mechanism itself. A lone agent in a static setting stalls Can agents evolve beyond the constraints humans engineer?.
Much of the practical progress avoids the risk by changing what improves. Self-improving agents split into a slow loop that rewrites model weights and a fast loop that updates prompts, memory and tools. Recent gains cluster in the fast loop because those edits are cheap and reversible Do self-improving agents really split into two distinct loops?. A memory-only agent adapted continually and reached 87.88% on GAIA validation without touching parameters Can agents learn continuously from experience without updating weights?. Swapping the execution harness around frozen weights lifted several models on Terminal-Bench, and the same runbook transferred to newer models Can execution harnesses lift model performance without retuning weights?. Reliability here comes from moving memory, skills and protocols into the structure around the model, so it doesn't have to re-solve them each time Where does agent reliability actually come from?. Reversibility is a safety net, not a standard, though. An outside signal still has to decide which edit to keep.
So agents can't improve reliably with no outside standard. What the corpus shows working is a standard that pushes back on the agent, sits outside its own head, and moves faster than it can be gamed. The evidence for evaluators that co-evolve with the agent comes from writing and proof tasks, so how far it extends to other domains is still open.
Sources 12 notes
Pure self-improvement stalls due to the generation-verification gap, diversity collapse, and reward hacking. Reliable improvement methods succeed by smuggling in external anchors: past model versions, third-party judges, user corrections, or tool feedback.
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
Apodex 1.1 demonstrates that agents trained via expanded executable environments and learned coordination behavior reach leading performance on complex professional tasks, even with smaller models. The approach achieves verifiable progress by treating state, recovery, and delivery as trainable behaviors rather than architectural requirements.
Across 17 frontier models on 36 expert-curated optimization tasks, repeated benchmark-edit-incorporate cycles within a wall-clock budget proved the dominant success predictor. Most models terminated early or burned budget unproductively; Claude Opus 4.6 stood out as persistent.
Static benchmarks saturate and invite gaming as agents strengthen. RQGM solves this by splitting search into epochs with fixed criteria per epoch but evolving objectives across boundaries, keeping improvement guarantees while moving the target faster than agents can exploit it.
Show all 12 sources
ALE's analysis of 960 real occupational workflows shows agents excel at abstract contests but fail long-horizon professional tasks. The gap is not model capability but benchmark design—the field optimizes what it measures, and it has measured contests rather than work.
Red Queen Gödel Machine makes evaluation part of the improvement loop, allowing agents to optimize writing and proof generation without a static verifier. Co-evolved systems match fixed-evaluator performance while using fewer tokens, suggesting shared learning drives efficiency.
A survey framework organizes co-evolving systems into three stages that progressively remove human engineering: dynamic peers first, then adaptive environments and feedback, finally the evolution mechanism itself. Single-entity self-improvement stalls in static contexts; co-evolution supplies adaptive pressure across multiple components.
A survey framework organizes self-improving agents into two update mechanisms: slow parametric loops updating foundation model weights, and fast non-parametric loops updating prompts, memory, and tools. Recent progress concentrates in the fast loop because scaffold updates are cheaper and reversible than weight updates.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Self-Improvements in Modern Agentic Systems: A Survey
- The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
- Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement
- Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design
- Hyperagents
- RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
- Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
- StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling