When an AI agent rewrites its own instructions from experience, is it learning general lessons or just patching past mistakes?

This explores what happens when an AI agent's scaffolding (the prompts, tools and instructions around a frozen model) is rewritten automatically from its own experience: do the edits add up to general lessons, or are they patches for the specific tasks the agent already saw? The corpus's most direct answer is that they are mostly patches. An analysis of evolved harness trajectories found the edits were rational and well-motivated across both prompt and tool layers, but most of them Do harness edits learn reusable strategies or memorize task fixes? persist fixes the agent could have rediscovered in a single rollout. The harness works like a cache. It stores what the agent could already reach, and it rarely turns a hard failure into a success. Because each edit reads as sensible, this is easy to miss by reading the edit log.

The corpus suggests this may depend on how edits are made rather than being a ceiling. SkillOpt treats a skill document as trainable external state for a frozen agent, and it only Can skill documents be optimized like neural network weights? accepts an edit if it improves performance on held-out validation tasks. That gate is a direct memorization test, since an edit that only patches one seen task shouldn't help on unseen ones. Its ablations go further: Does constraining edits make skill learning more stable? a textual learning-rate budget, the validation gate and a buffer that keeps rejected edits as negative feedback give more stable and more generalizable improvement than letting the agent rewrite its own instructions freely. The two findings come from different systems, and the corpus has no head-to-head comparison. Still, they fit together: unconstrained self-editing drifts toward memorization, and constraints push it toward strategy.

Who does the evolving, and who reads the result, also matters. The ability to produce useful harness edits Do stronger models always evolve harnesses better? is roughly flat across model tiers. The ability to benefit from those edits follows an inverted U that peaks in mid-tier models. Weak models fail to invoke the harness at all, and strong models struggle to follow it faithfully. So a good edit can still fail to help. My own inference, which the notes don't state, is that strong models may also gain little from task-specific fixes because they could have found them anyway.

There is evidence that gains from this kind of optimization can travel. AIDE2's improvements Do AIDE2's improvements transfer to unseen tasks? held on four held-out benchmarks, including physics-based weather forecasting, which sits outside the task distribution used to select its improvements. That is the test the memorization finding calls for: score on tasks the optimizer never saw.

Two older designs show what reusable knowledge looks like when it is built for reuse. Voyager stores executable skills in an indexed library and composes complex skills from simpler ones, so Can agents learn new skills without forgetting old ones? knowledge compounds instead of piling up as one-off fixes. Reflexion keeps verbal reflections in episodic memory, and it works because Can agents learn from failure without updating their weights? the success-or-failure signal is unambiguous, which stops the agent from rationalizing. Reflexion's memory is openly tied to specific episodes. The practical question is therefore whether an edit could be rediscovered in one rollout and whether it survives a held-out check. If the answer is yes to the first or no to the second, it is memorization.