Is a self-improving AI fundamentally different from a model that just keeps getting better at a fixed task, or is it the same loop with two settings flipped?

This explores what separates an AI that improves *itself* in an open-ended loop from the more familiar cycle where a policy gets better against a fixed goal. The corpus's sharpest answer is that they are less different than they sound. Both run the same improvement cycle. What separates them is two settings, or 'dials': whether the thing doing the improving sits inside the agent, and whether the standard for 'better' comes from outside it What separates self-improvement from policy improvement?. Iterative policy improvement keeps the improver and the scorecard external. Recursive self-improvement moves both inward, so the system grades and rewrites itself.

That second dial is where the trouble starts. When a model judges its own progress, the loop becomes circular. Models are often better at producing answers than at checking them, outputs lose variety over successive rounds, and the system learns to game its own reward. The self-improvement methods that work turn out to quietly bring in an outside reference point: an earlier model version, a third-party judge, user corrections, or feedback from tools Can models reliably improve themselves without external feedback?. So most of what gets called 'self-improvement' today is closer to policy improvement with an outside reference point than to true recursion. A large survey makes the same split from the industry side. Bounded self-refinement, where you can measure whether the output got better, is routine practice. Open-ended recursive self-improvement is a different thing, held back by the need for real-world grounding, by collapse dynamics, and by compute Are self-refinement and recursive self-improvement actually the same thing?.

The first dial, where the improver sits, has a more practical side. Self-improving agents can update themselves in two ways. One is a slow loop that retrains the model's weights. The other is a fast loop that rewrites prompts, memory, and tools. Most recent progress is in the fast loop because those changes are cheap and easy to undo Do self-improving agents really split into two distinct loops?. Lilian Weng argues that recursive self-improvement will start there too. It moves from instruction prompts, to the code wrapped around the model (the 'harness'), to the optimizer code itself, well before any model rewrites its own weights Does recursive self-improvement start with harness engineering?. Sakana AI frames the problem as learning more from fewer tries rather than using more compute Can sample efficiency replace compute scale in recursive self-improvement?.

The less obvious point is that the deepest difference may not be who grades the work but who sets the goal. One participant in a debate on the topic argues that fast recursive self-improvement depends on AIs proposing their own research objectives and pursuing them without drifting Can AIs learn to specify their own research objectives?. Policy improvement never has to do this, because the objective is given. A related line of work says that agents automating R&D improve what they produce while the research process itself stays just as efficient. Only an agent that rewrites its own code changes that Can recursive self-improvement speed up the research process itself?.

Current evidence doesn't show sustained recursion. One run with seven accepted self-rewrites doesn't report how large each gain was, so it can't show whether returns kept coming or tapered off Does recursive self-improvement sustain gains or hit diminishing returns?. A simple model of the feedback loops says acceleration depends on how strong each loop is, multiplied together. Today's loops are getting stronger but can't yet sustain themselves Are AI feedback loops strong enough to sustain recursive self-improvement?. That gap is why Anthropic has urged caution about the moment the dials turn all the way inward Does recursive self-improvement pose serious risks to society?.