Should you teach an AI agent by retraining its brain, or by editing its notes, tools, and instructions?
How do parametric and non-parametric updates differ in agents?
This explores the difference between teaching an agent by changing the model's weights (parametric) and teaching it by changing everything around a frozen model, such as its prompts, memory, skills, and tools (non-parametric).
This explores the difference between teaching an agent by changing the model's weights (parametric) and teaching it by changing everything around a frozen model, such as its prompts, memory, skills, and tools (non-parametric). In the corpus, the two differ less in what they can achieve than in speed, cost, and how easily you can undo a mistake.
One survey framing splits self-improving agents into two loops: a slow one that updates foundation model weights and a fast one that updates prompts, memory, and tools Do self-improving agents really split into two distinct loops?. Most recent progress sits in the fast loop because scaffold edits are cheaper and reversible. A bad memory entry can be deleted, while a bad weight update means retraining.
The fast loop can do more than you might expect. StateM tuned the execution system around frozen models and lifted Terminal-Bench 2.1 accuracy across several of them, reaching 95.3% on GPT-5.6. The same runbook carried over to newer models unchanged, which a weight update cannot do Can execution harnesses lift model performance without retuning weights?. AgentFly treats learning as memory operations (stored cases, subtasks, and tool use) and reached 87.88% on GAIA validation without touching the LLM's parameters Can agents learn continuously from experience without updating weights?. One explanation for why this works is that reliable agents move memory, skills, and interaction protocols out of the model and into a harness, so the model doesn't have to solve the same problems repeatedly Where does agent reliability actually come from?.
The weight side is more targeted than it sounds. RL training changes only 5–30% of parameters, and those sparse updates are nearly full-rank and nearly identical across random seeds. That points to a consistent subnetwork being adjusted, not diffuse rewriting Does reinforcement learning update only a small fraction of parameters?. The corpus has no head-to-head comparison of the two kinds of update on the same task. What it does show is that they suit different timescales. MetaClaw pairs them: skills injected from failures in seconds with zero downtime, and gradient optimization run in idle windows. Each feeds the other, because better policies produce more informative failures and richer skills produce higher-reward trajectories Can agents adapt without pausing service to users?.
Non-parametric updates have two limits. The first is that a scaffold only helps if the model uses it. Models are equally able to write useful harness edits at every tier, but the benefit from those edits peaks in mid-tier models. Weak models fail to invoke the harness, and strong ones struggle to follow its instructions faithfully Do stronger models always evolve harnesses better?. The second is that the design of the memory matters. Reconstructing memory by traversing a graph while reasoning gained up to 23% over retrieve-then-reason pipelines Can agents reconstruct memory on demand instead of retrieving it?. Even without weight updates, how the agent stores and retrieves what it learned makes a large difference.
Sources 8 notes
A survey framework organizes self-improving agents into two update mechanisms: slow parametric loops updating foundation model weights, and fast non-parametric loops updating prompts, memory, and tools. Recent progress concentrates in the fast loop because scaffold updates are cheaper and reversible than weight updates.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
Across seven RL algorithms and ten LLM families, RL induces intrinsic parameter sparsity of 5–30% without explicit regularization. Critically, these sparse updates are nearly full-rank and nearly identical across random seeds, indicating structural rather than arbitrary parameter selection.
Show all 8 sources
MetaClaw demonstrates that deployed agents require both rapid skill injection from failures (seconds, zero downtime) and slower gradient-based optimization during idle windows (minutes to hours). The two mechanisms reinforce each other, with better policies producing more informative failures and richer skills enabling higher-reward trajectories.
Model capability to produce useful harness edits stays constant across tiers, but capacity to actually benefit from those edits follows an inverted U-shape, peaking in mid-tier models. Weak models fail to invoke harnesses; strong models struggle with faithful instruction-following.
MRAgent achieves up to 23% gains on reasoning tasks by reconstructing memory through active graph traversal that prunes paths based on accumulated evidence, while reducing token and runtime cost compared to fixed-retrieval pipelines.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- Useful Memories Become Faulty When Continuously Updated by LLMs
- Rethinking the Evaluation of Harness Evolution for Agents
- RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
- AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
- ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
- StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization