Can self-distillation reduce catastrophic forgetting in continual learning?
This reads the question as: does training a model on its own outputs (self-distillation) help it learn new tasks without overwriting old ones — and the corpus actually pushes back on the premise.
This explores whether self-distillation — training a model on its own generated outputs — is a good tool against catastrophic forgetting, and the most direct evidence in the collection is a caution, not an endorsement. The one note that studies self-distillation head-on finds it can quietly degrade a model: by training on its own confident traces, the model stops producing the uncertainty markers ('Wait', 'Hmm') that flag flawed reasoning, trading robustness for confident brevity and losing the ability to self-correct on unfamiliar problems Does self-distillation harm mathematical reasoning performance?. A closely related finding shows the same trap from the teacher side: when the teacher is conditioned on correct answers, the distilled student inherits confident, concise traces that perform well in-domain but generalize worse out-of-distribution Does richer teacher context hurt student generalization?. So self-distillation isn't a neutral memory-preservation move — it actively reshapes what the model keeps and discards.
What makes this interesting is that the corpus reframes catastrophic forgetting itself. Forgetting isn't an inherent cost of learning new things — it's a misallocation problem. Fast-Slow Training shows that if you route task-specific lessons into the prompt (fast, textual context) and keep parameter updates minimal (slow weights), you get equivalent performance with substantially less forgetting Can splitting adaptation into two channels reduce forgetting?. The mechanism behind why this works also clarifies the self-distillation risk: staying close to the base model's distribution — low KL drift — is what preserves the model's plasticity to keep learning. Models that drift far stall when the task domain shifts Does staying close to the base model preserve learning ability?. Self-distillation, by sharpening the model toward its own confident outputs, tends to increase drift in exactly the direction that costs you future adaptability.
The collection's strongest anti-forgetting strategies all share a different instinct: don't update the weights you want to protect. SoftCoT freezes the main LLM entirely and delegates new reasoning to a small auxiliary model, so pre-trained knowledge can't be overwritten Can continuous reasoning avoid forgetting in instruction-tuned models?. VOYAGER stores new skills in an external, executable library and composes them rather than baking them into weights Can agents learn new skills without forgetting old ones?. AgentFly goes furthest, achieving continual adaptation entirely through episodic memory operations with zero parameter updates Can agents learn continuously from experience without updating weights?. The pattern is consistent — the safest place to put new knowledge is somewhere other than the weights that already hold the old knowledge.
There's one nuance worth pulling out, because it's where self-distillation could plausibly earn its keep. The failure of self-correction training isn't that models learn from themselves — it's that they learn from the *wrong* distribution. Training on offline correction traces fails because those errors don't match the model's actual test-time errors; multi-turn online RL on the model's own live mistakes succeeds Why does self-correction training on offline data fail?. The lesson transfers: self-generated data is dangerous when it's confident, sanitized, and off-distribution, but useful when it's grounded in the model's real behavior — which is also why bidirectional RAG only writes self-generated answers back into its corpus after entailment and novelty checks Can RAG systems safely learn from their own generated answers?.
So the honest answer the corpus supports: self-distillation, as studied here, is more likely to *cause* a subtle form of forgetting — the loss of self-correction and out-of-distribution robustness — than to cure catastrophic forgetting. If your goal is continual learning without forgetting, the collection points you toward separating the storage of new knowledge from old (frozen backbones, fast context channels, external memory) and toward keeping KL drift low, rather than toward distilling the model into a more confident version of itself.
Sources 9 notes
Self-distillation reduces performance in mathematical reasoning by eliminating epistemic markers like "Wait" and "Hmm" tokens that flag flawed reasoning paths. These tokens enable self-correction on out-of-distribution problems, so removing them sacrifices robustness for confident brevity.
Teachers conditioned on correct answers and verifier output produce confident, concise traces that students inherit. This style suppresses uncertainty expression, optimizing in-domain performance while degrading generalization to out-of-distribution problems that require epistemic caution.
Fast-Slow Training routes task-specific lessons into optimized prompts while keeping parameter updates minimal, reaching equivalent performance 1.4–3x faster with substantially less catastrophic forgetting and plasticity loss, demonstrating that forgetting is a misallocation problem rather than an inherent cost.
FST-trained models stay up to 70% closer to their base distribution than parameter-only RL, and this reduced drift preserves the model's ability to learn subsequent tasks effectively. Parameter-only approaches stall when task domains change, while low KL drift enables sustained adaptation.
SoftCoT avoids catastrophic forgetting by keeping the main LLM frozen while delegating soft thought generation to a small auxiliary model. This architectural separation maintains pre-trained knowledge while enabling continuous reasoning.
VOYAGER demonstrates that storing executable skills in an embedding-indexed library and composing complex skills from simpler ones allows agents to learn continuously while avoiding the forgetting that occurs with weight-update-based methods. Environmental feedback refines skills while an automatic curriculum drives continual exploration.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
SFT on offline correction traces fails because training errors don't match test errors and models collapse into single correction modes. Multi-turn online RL under the model's own error distribution successfully trains self-correction by letting models practice correcting their actual mistakes.
Systems can add generated answers to their retrieval corpus when outputs pass entailment verification, source attribution checks, and novelty detection. This prevents hallucinations from polluting future retrievals while allowing genuine knowledge accumulation.