Give an AI a perfect memory of everything it's done, and it can end up learning worse than one starting fresh.

This explores why a model that carries a growing store of past experience can end up learning worse than one that starts fresh each time. The corpus suggests this is a repeatable result. CL-BENCH found that plain in-context learning, with no memory system at all, beat dedicated memory architectures on most domains, and the best memory system gained only 25% over a stateless baseline Do memory systems actually help language models learn continuously?. MemTrapBench is starker. All five memory frameworks it tested fell below a no-memory baseline, by more than 10%, even though the stored memories were accurate and relevant to the task Can relevant memories actually harm LLM reasoning?.

That second result matters because the memories weren't wrong. The damage happens when the memory is used. A model with no memory has to reason from the problem in front of it. A model with memory is handed a prior that competes with that problem. CL-BENCH names the symptoms as spurious generalizations and stale beliefs, meaning lessons from earlier tasks that get applied where they don't fit. Standard memory benchmarks miss this failure because they check whether the right thing was stored, not whether using it helped.

Accumulation also compounds the problem when the memory is rewritten. When an LLM keeps consolidating its experience into textual summaries, utility follows an inverted U. It helps early, then declines as experience piles up, and eventually falls below simply keeping the raw episodes. In one test GPT-5.4 failed 54% of problems it had previously solved after consolidation Does agent memory degrade when continuously consolidated?. The paper identifies three mechanisms: misgrouping (lumping different situations together), applicability stripping (keeping the lesson but losing the conditions under which it holds), and overfitting to narrow streams of experience. A rule that survives without its 'when this applies' is worse than no rule.

The neighbouring work suggests fixes along three lines. One is where memory lives. The Sleep paradigm and recurrence-based consolidation both move recent context into weights during offline passes, so consolidation is separated from the moment of answering Can models consolidate memories during offline sleep phases? Can recurrence consolidate memory without predicting tokens?. Another is what form memory takes. VOYAGER stores executable skills that are refined by environmental feedback and composed into bigger skills, which is a different kind of memory from a paragraph of lessons learned Can agents learn new skills without forgetting old ones?. The third is when memory gets used. DeepRAG learns per reasoning step whether to retrieve or rely on what the model already knows, and its 22% gain comes largely from cutting the noise of unnecessary external knowledge When should language models retrieve external knowledge versus use internal knowledge?.

Moving memory into weights doesn't make the problem go away. No single continual-learning mechanism held strong retention across 100 sequential tasks Can any single continual learning method prevent forgetting?. Models that drift far from their base distribution also lose the ability to learn the next task, while staying closer preserves it Does staying close to the base model preserve learning ability?. So the useful question isn't memory versus no memory. Whatever accumulates, in text or in weights, has to earn its place, and a stateless model is a strong baseline because it can't be misled by its own past.