Could you unplug an AI's memory from one model and plug it into a different one, or is it stuck?
Can a memory module be swapped between different base models?
This explores whether a memory component (a separate module that stores what an AI has learned) can be unplugged from one language model and used with another, or whether it stays tied to the model it was built with.
This explores whether a memory component can be unplugged from one language model and used with another, or whether it stays tied to the model it was built with. The corpus has no note that tests a direct swap, so what follows is inference. The notes do sort memory designs by how tightly the memory is bound to the model, and that binding is what decides how portable it is.
The most swappable designs keep memory outside the model's weights. Can a separate memory model inject knowledge without touching the LLM? trains a dedicated memory model to hold new knowledge and works with frozen proprietary models, so the main LLM is never touched. The cost is up-front training and limited capacity. Can external state caches let models solve harder problems? goes further and keeps state in a persistent REPL (a live code environment) and in disk-backed history. That state sits outside the model entirely, and the authors say this lets them tell harness failures from model failures. That separation is what a swap needs: you can blame the memory or the model, not both at once.
The middle ground is a small trained component sitting next to a frozen model. Can lightweight adapters replace millions of personalized models? treats adapters as durable state that lets one base model serve millions of users. An adapter is a small set of weight adjustments, so it is portable across users but presumably not across bases. It is defined relative to the weights it modifies, so a new base would likely need a new adapter. Can continuous reasoning avoid forgetting in instruction-tuned models? has the same shape, with a small helper model feeding a frozen backbone. In both, swapping the base probably means retraining the small piece, not the big one. That is much cheaper than retraining everything, but it is not plug-and-play.
The least swappable design is also the one with the strongest argument against swapping. Should agent memory live inside the model backbone? builds memory into the model backbone, because an external memory and a backbone that optimize separately can drift apart and fail to work together. Portability and co-training pull in opposite directions. The memory-as-parameters result in Does allocating parameters to memory beat scaling the base model? also describes memory paired with a particular base, not memory moved between bases. A 6.9B memory with a 410M base beat a 12B base-only model. The note does not say whether that memory would still help a different base.
Two other notes suggest a swapped-in memory has to clear a higher bar than plugging in. Do memory systems actually help language models learn continuously? finds that dedicated memory systems often lose to simply keeping examples in the prompt. Accumulated state also picks up stale beliefs and spurious generalizations, and a memory carried to a new model would carry those along. Does agent memory work better at one level of abstraction? adds that the right memory format (reusable workflows, causal rules, or state-action records) depends on the task domain. So a memory can fit the model but not the task.
The closest evidence for carrying something across models is Can a stronger model lift a weaker one at test time without retraining?. A stronger model built inference-time harnesses that nearly doubled a weaker model's score without retraining. The harnesses worked mainly by moving fragile reasoning into deterministic code and task-specific routing. That is not memory, but it points the same way: what transfers between models is what lives in code and structure outside the model, not what is baked into weights.
Sources 9 notes
MeMo trains a dedicated memory model to encode new knowledge, eliminating inference-time search costs that scale with corpus size. It avoids fine-tuning risks and works with frozen proprietary models, but trades this for up-front training cost and capacity limits.
Prime Agent organizes persistent state in four levels (weights, context, persistent REPL plus subagents, disk-backed history) to let models read and write addressable state beyond their instruction stream. The approach isolates harness failures from model failures and reported gains on ARC-AGI-3, though specific components remain unablated.
PEFT adapters function as durable behavioral deltas carrying learned user experience, enabling a single strong base plus millions of lightweight adapters to replace millions of full models—but only when scale-up, scale-down, and scale-out reinforce simultaneously.
SoftCoT avoids catastrophic forgetting by keeping the main LLM frozen while delegating soft thought generation to a small auxiliary model. This architectural separation maintains pre-trained knowledge while enabling continuous reasoning.
Metis demonstrates that agent memory can be implemented as a persistent state and autonomous procedures within the model backbone rather than external modules. This approach enables end-to-end training and avoids the decoupling failures where external memory and backbone optimize independently.
Show all 9 sources
A 6.9B memory paired with a 410M base model outperformed a 12B base-only model on 17 benchmarks while using 39% fewer total parameters. This result held across multiple model scales and domain-adaptation tasks, suggesting memory is a more efficient capacity axis than base-model growth.
CL-BENCH's gain metric isolates true learning from base capability and finds that naive in-context learning outperforms dedicated memory architectures on most domains, with the best system gaining only 25% over a stateless baseline. Accumulated state introduces spurious generalizations and stale beliefs.
Workflow-level memory wins in routine-rich domains, causal-rule memory in environment-rich domains, and state-action memory in spatially-rich web tasks. The optimal abstraction depends on whether task variance comes from arguments, causal structure, or fine-grained UI state.
A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Know It, Act on It: Investigating Memory Utilization in LLM Personalization
- SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs
- Are We Ready For An Agent-Native Memory System?
- Useful Memories Become Faulty When Continuously Updated by LLMs
- Provable Benefits of In-Tool Learning for Large Language Models
- Metis: Memory Foundation Model
- Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
- MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use