What separates self-improvement from policy improvement?

The paper opens by asking whether recursive self-improvement (RSI) is "a phenomenon, a mechanism, or a prospect," and observes that RSI is "being claimed at many scales" with no single formal framework describing the instances. Its classical counterpart, generalized policy iteration (GPI), has "well-understood theoretical properties, but only where the update principle and the evaluation base lie outside the agent." Generalized Agent Iteration (GAI) is proposed to describe iterative policy improvement and RSI "as two cases of a single learning paradigm." The introduction supplies the range of claims the framework has to cover, from agents that refine their own outputs, through systems that rewrite the routine that improves them, and research loops that search programs or experiments under fixed evaluators, to systems that co-evolve the standard they are judged by or "propose to do without one."

The mechanism is compact. An agent is "a configuration of modifiable components within a system," and learning is a cycle of agent evaluation and agent improvement. Two "pivotal dials" then distinguish the instances. The first is whether the improving mechanism is part of the agent, and it "delineates the boundary between GPI and RSI." The second is whether the standard the agent is measured against is grounded outside it, and it fixes a system's polarity as "anchored, goal drift, or fully self-referential." The excerpt names the three polarities but does not define them beyond that.

The dials cut across the neighbors rather than repeating them. The survey framing in Do self-improving agents really split into two distinct loops? also treats the agent as a configuration and self-improvement as an update operator, but it sorts systems by what gets updated, weights or scaffold. GAI sorts them by who runs the update and against what. The second dial gives structure to the claim in Can models reliably improve themselves without external feedback?: that note argues from evidence that reliable fixes need external anchoring, and GAI makes anchoring a variable with three settings instead of a verdict. The paper's treatment of a "proof gate of the Gödel machine or a coding benchmark" as external guards on what a modifier may propose is the frame in which Can AI systems improve themselves through trial and error? can be read: swapping proofs for benchmarks changes the guard, not the fact of having one. And the premise in Can recursive self-improvement speed up the research process itself? reads as a statement about the first dial, an improver that sits outside the thing it improves. That last mapping is my reading, not the paper's.

The excerpt establishes a vocabulary and no result. It contains no theorem, experiment or mapping of named systems onto dial settings, and the paper says as much: the framework "describes how a system changes, not how to compute the change," carries no reachability or complexity claims, does not model time scales, and cites rather than measures the defects it attributes to existing systems. Whether a guard should itself be an agent component is "a design choice we leave open." The passage also announces four limits and states three. What follows at this strength is a discipline for reading self-improvement claims: ask whether the improver is inside the agent and whether the standard is grounded outside it. Which configurations work, or fail, is not something this excerpt can say.