An AI's underlying model can't be retrained on the job, so does the way it keeps notes decide whether it improves?
What shapes of memory help frozen agents improve without retraining?
This explores what form an agent's external memory should take (raw reflections, compressed summaries, causal rules, skill libraries, linked graphs) so that a model whose weights never change can still get better over time.
This explores what form an agent's external memory should take (raw reflections, compressed summaries, causal rules, skill libraries, linked graphs) so that a model whose weights never change can still get better over time. The corpus suggests the form matters more than the amount: what you store, and how you shape it, largely decides whether a frozen agent improves or just accumulates clutter.
The simplest shape is a plain-language diagnosis of what went wrong. Reflexion-style agents write a verbal self-critique after each failed attempt and store it as episodic memory, and they improve across episodes with no parameter updates Can agents learn from failure without updating their weights?. Two details make it work. The feedback is a clear success or failure, which stops the agent from rationalizing. And the reflections stay uncompressed, so they remain usable. The limit is that a generic reflection like "I should be more careful" carries little. A sharper shape keeps the conditions under which a lesson applies. Causal-form memory does this, and it beat generic reflection by 23 points on repeated trials. It also gained 4-17 points when transferred to new environments Can frozen language models continually improve through memory structure alone?. A lesson that says when it applies can be reused somewhere new. A lesson that only says what happened can't.
A second family stores things the agent can do, not things it noticed. VOYAGER keeps executable skills in an embedding-indexed library, and new skills are composed from simpler ones. That lets it keep learning without the forgetting that weight updates cause Can agents learn new skills without forgetting old ones?. AgentFly stretches the idea into a full learning loop with case, subtask and tool memories. It does credit assignment and policy improvement entirely through memory operations, reaching 87.88% on GAIA validation with the model untouched Can agents learn continuously from experience without updating weights?. Skill libraries have their own failure mode, which is that they fill up with generic, wordy entries. SkillOS finds that a separately trained curator, paired with a frozen executor, pushes the library toward actionable execution logic and cross-task strategies, and it generalizes across different executor models Can a separate trained curator improve skill libraries better than frozen agents?. So the agent can be frozen, but someone has to look after the memory.
A third question is whether memory should stay static or keep changing. FluxMem treats memory as a network whose links are created, refined and pruned by execution feedback, and it reaches state-of-the-art on three benchmarks Should agent memory adapt dynamically based on execution feedback?. MRAgent moves the reasoning into retrieval. It rebuilds memory as it walks a graph, dropping paths that the evidence doesn't support, and it gains up to 23% while costing less than retrieve-then-reason Can agents reconstruct memory on demand instead of retrieving it?. For long histories, DeepAgent's memory folding lets the agent compress its own past into episodic, working and tool schemas. This cuts tokens and lets it stop and rethink strategy, and the compression works because the agent chooses when to fold and the result has structure Can agents compress their own memory without losing critical details?.
The lateral question is where the memory sits, and how frozen the agent has to be. AdaCoM keeps the agent frozen but trains an external manager to prune its context. It finds the best policy depends on the agent: strong agents do better with high-fidelity context, and weak ones need aggressive compression Can an external manager handle context for frozen agents?. MeMo goes further and trains a separate memory model to hold new knowledge. It removes search costs that grow with corpus size and works with proprietary frozen models, at the price of up-front training and limited capacity Can a separate memory model inject knowledge without touching the LLM?. Metis takes the opposite view, that memory should be a native state inside the backbone so memory and model are trained together instead of drifting apart Should agent memory live inside the model backbone?. That last one is arguably no longer "frozen", so it works as the counterpoint. A curious side finding is that RL agents can even use their physical surroundings as memory without being told to Do RL agents accidentally use environments as memory?.
The pattern across these notes is that useful memory keeps its applicability conditions, is executable or actionable instead of vague, and is reshaped by feedback. The corpus is thinner on how these shapes compare head-to-head under the same benchmark, so which one wins for a given task is still open.
Sources 12 notes
Agents using causal-form memory (preserving applicability conditions) outperform generic reflection by 23 points on repeated trials and gain 4-17 points transferring to new environments, showing memory shape matters more than parameter updates.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
DeepAgent's autonomous memory folding consolidates interaction history into episodic, working, and tool memory schemas. This reduces token overhead while letting agents pause to reconsider strategies—the autonomy and structure together avoid degradation that plagues poorly designed consolidation.
VOYAGER demonstrates that storing executable skills in an embedding-indexed library and composing complex skills from simpler ones allows agents to learn continuously while avoiding the forgetting that occurs with weight-update-based methods. Environmental feedback refines skills while an automatic curriculum drives continual exploration.
Mathematical proof shows that environmental artifacts reduce information needed to represent history in RL agents. Path-following agents naturally develop memory-like behavior through standard reward optimization, satisfying situated cognition criteria without explicit memory objectives.
Show all 12 sources
FluxMem demonstrates that adaptive memory topology—where links form, refine, and consolidate based on closed-loop execution feedback—consistently reaches state-of-the-art across three distinct benchmarks. Dynamic connectivity outperforms fixed retrieval by aligning abstraction and eliminating interference.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
SkillOS shows that separating a trainable curator from a frozen executor, grouped by task streams, causes skill repositories to shift from generic verbose additions toward actionable execution logic and cross-task meta-strategies. The trained curator generalizes across different executor backbones and domains.
MRAgent achieves up to 23% gains on reasoning tasks by reconstructing memory through active graph traversal that prunes paths based on accumulated evidence, while reducing token and runtime cost compared to fixed-retrieval pipelines.
Metis demonstrates that agent memory can be implemented as a persistent state and autonomous procedures within the model backbone rather than external modules. This approach enables end-to-end training and avoids the decoupling failures where external memory and backbone optimize independently.
AdaCoM trains an external RL-based manager to prune and preserve context for frozen agents. The key finding: stronger agents benefit from high-fidelity preservation, while weaker agents need aggressive compression—optimal context management is agent-specific, not task-universal.
MeMo trains a dedicated memory model to encode new knowledge, eliminating inference-time search costs that scale with corpus size. It avoids fine-tuning risks and works with frozen proprietary models, but trades this for up-front training cost and capacity limits.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Useful Memories Become Faulty When Continuously Updated by LLMs
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization
- Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
- Rethinking Memory as Continuously Evolving Connectivity
- AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
- SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
- Are We Ready For An Agent-Native Memory System?
- SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning