Does harness self-improvement memorize tasks instead of learning broadly?

The paper starts from the observation that an agent's capability is "largely magnified by its harness," meaning the prompts, control flow, tooling, memory, and context management around a frozen backbone model. Recent methods automate harness engineering by having LLMs propose and select component-wise edits from task feedback, which the authors describe as "a form of recursive self-improvement (RSI) at the agent-system level." Their warning is that this loop "may overfit by memorizing the training tasks, showing large in-distribution gains that shrink or even vanish on out-of-distribution benchmarks." RRSI (Regularized Recursive Self-Improvement of Agent Harnesses) is offered as the response: bring "the principles of regularizations" into harness evolution.

The regularization is applied to the two halves of the evolution loop rather than to the model. The proposer works under a "temporally annealed budget" that limits how many edits one candidate can bundle, and it is pushed toward "unexplored trajectories based on evolution history." The selector gets two components. A critic screens out benchmark-specific proposals, and a pruner removes changes that are "too small, too expensive, or no longer useful." The stated aim is that these constraints "favor reusable agent mechanisms over benchmark-specific ones or even noises." The framing treats an accepted harness edit like a learned parameter: without a penalty on complexity and specificity, the search keeps whatever raises the evolve-set score, whether or not it generalizes.

This sits next to the survey framing in Do self-improving agents really split into two distinct loops?. RRSI works entirely in the fast loop and names a failure mode of it: scaffold edits are cheap to make and cheap to accept, so nothing in the loop by default asks whether an edit transfers. It also adds a different axis to Do stronger models always evolve harnesses better?. That note asks who can produce and use updates; this paper asks whether an update that helped on the evolve tasks helps anywhere else. The two are separable, though the excerpt only says transfer was evaluated "across multiple domains, benchmarks, and policy models" and reports nothing about how the results split by model. There is also a scope contrast with What predicts success in ultra-long-horizon agent tasks?, where more feedback cycles predict success on a task. RRSI's concern is the reverse case, where more cycles over a finite task set may fit that set. The settings differ, so these are not a direct conflict.

The excerpt is only the abstract, one introduction passage, and a limitations passage, and it establishes little beyond the design and its motivation. It gives no gain figures, no size for the in-distribution versus out-of-distribution gap, no benchmark names, and no ablation showing which of the annealed budget, history-based exploration, critic, or pruner does the work. The overfitting problem is stated as something that "may" happen, and the claim that the constraints favor reusable mechanisms is a design intent in the abstract, not a result the excerpt shows. The authors themselves list limits: frozen backbones only, a finite evolve set, several regularization hyperparameters, dependence on feedback quality and search budget, and a need for "broader validation" on different agent architectures, tool ecosystems, and longer self-improvement runs. What follows at this strength is a caution: an unregularized harness-evolution loop that reports only in-distribution gains should be read as unverified, and the choice of which edits a selector may keep is a design decision of its own.