Can an AI get smarter by rewriting its own notes and tools, or only by retraining its brain?
What distinguishes scaffold-level changes from parametric weight updates in self-improvement?
This explores the difference between an agent improving by rewriting the things around the model (prompts, memory, tools) and improving by changing the model's own weights.
This explores the difference between an agent improving by rewriting the things around the model (prompts, memory, tools) and improving by changing the model's own weights. The corpus says the split is real, and it mostly comes down to speed, cost, and how easily you can undo a change. The deeper limits on both turn out to be the same.
The clearest framing is two loops. A slow parametric loop updates foundation model weights, and a fast non-parametric loop updates prompts, memory, and tools Do self-improving agents really split into two distinct loops?. Recent progress concentrates in the fast loop because scaffold updates are cheaper and reversible. A bad prompt edit can be rolled back, but a bad training run has already reshaped the model. Weight updates are also more structured than they look. RL touches only 5 to 30 percent of parameters, in sparse but nearly full-rank subnetworks that come out nearly identical across random seeds Does reinforcement learning update only a small fraction of parameters?. So "update the weights" isn't one blunt operation either.
The line between the two is also blurrier than a two-loop diagram suggests. ReFT leaves the weights frozen and learns small interventions on the hidden representations instead. Its low-rank variant beats LoRA while using 10 to 50 times fewer parameters Can editing hidden representations beat weight updates for finetuning?. That changes how the model computes without changing what the model is. It suggests the real axis is how deep and how permanent an edit is, with prompts at one end and full weight training at the other.
What doesn't change across these choices is the failure mode. Reward hacking appears when weights are trained, when outputs are selected, and when prompts are revised. The shared cause is optimizing against a signal that only partly captures the task Does reward hacking always stem from the same failure?. The ceiling is shared too. Pure self-improvement is circular, and the methods that work smuggle in external anchors such as past model versions, third-party judges, user corrections, or tool feedback Can models reliably improve themselves without external feedback?. The formal version is that a model can only improve where it verifies better than it generates What limits how much models can improve themselves?. Cheap scaffold edits let you try many more variations, but something still has to score each one. When that judge stays fixed, loops plateau. Co-evolving the judge alongside the actor lifted AlpacaEval 2 from 23% to 39% Why do self-improvement loops plateau without updating the judge?.
The corpus can't yet say whether the fast loop keeps paying off. One paper reports seven accepted rewrites in an 8-day run but gives neither the size nor the timing of each gain, so there's no evidence either way on diminishing returns Does recursive self-improvement sustain gains or hit diminishing returns?. A survey of 1,250 papers adds that bounded, checkable self-refinement, which is what industry does today, is a different phenomenon from open-ended recursive self-improvement Are self-refinement and recursive self-improvement actually the same thing?.
Sources 9 notes
A survey framework organizes self-improving agents into two update mechanisms: slow parametric loops updating foundation model weights, and fast non-parametric loops updating prompts, memory, and tools. Recent progress concentrates in the fast loop because scaffold updates are cheaper and reversible than weight updates.
Across seven RL algorithms and ten LLM families, RL induces intrinsic parameter sparsity of 5–30% without explicit regularization. Critically, these sparse updates are nearly full-rank and nearly identical across random seeds, indicating structural rather than arbitrary parameter selection.
ReFT learns task-specific interventions on frozen model representations rather than updating weights, with LoReFT (low-rank linear subspace variant) dramatically outperforming LoRA across reasoning, instruction-following, and NLU benchmarks while using far fewer parameters.
Reward hacking arises during weight training, output selection, and prompt revision through a shared failure: optimization against signals that incompletely represent the actual task. The substrate matters less than the misalignment between the scoring function and ground truth.
Pure self-improvement stalls due to the generation-verification gap, diversity collapse, and reward hacking. Reliable improvement methods succeed by smuggling in external anchors: past model versions, third-party judges, user corrections, or tool feedback.
Show all 9 sources
Models can only improve themselves when they verify solutions better than they generate them. This gap scales with model size but vanishes entirely for factual tasks, predicting which domains benefit from self-improvement.
Meta-Rewarding adds a meta-judge layer that evaluates the judge's own judgments, creating preference data for both actor and evaluator. This co-evolution improved AlpacaEval 2 from 23% to 39% and Arena-Hard from 21% to 29% without supervision.
The paper reports seven successive improvements in an 8-day run but provides neither the magnitude of each gain nor their timing. Without score trajectories and longer-horizon data, the evidence supports only that improvements transferred, not that recursive self-improvement sustains returns against diminishing curves.
A 1,250-paper survey shows that bounded, evaluable self-refinement (current industrial practice) differs fundamentally from open-ended recursive self-improvement, which remains constrained by grounding requirements, collapse dynamics, and compute limits measurable today.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
- Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement
- Self-Improvements in Modern Agentic Systems: A Survey
- Hyperagents
- Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
- Dream-RSI: Recursive Self-Improvement through Evolving Worlds
- The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
- PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents