Continual Learning Mechanisms Compose for Long-Horizon Memorization

Abstract. Language models may need to internalize information that arrives over time and retain it through many subsequent updates. To study this challenge, we introduce long-horizon memorization, a setting in which a model learns 100 query-answer tasks through continual supervised fine-tuning without retaining earlier training examples or receiving task identifiers at inference. Sequential updates cause catastrophic forgetting, and no single continual learning mechanism we evaluate maintains strong retention at this horizon. We hypothesize that mechanisms addressing complementary sources of forgetting will be more effective when composed. We organize these compositions along two design dimensions. Data, function, and weight anchors specify what prior information each update should preserve, while low-rank allocation rules determine where successive updates are retained. To test this hypothesis systematically, we construct three distinct 100-task memorization datasets. We introduce task-level successive halving to search the combinatorial design space and use a factorial experiment to measure individual and interaction effects.

Introduction. Consider a language model that learns new information over time by updating its parameters. For these parameters to serve as memory, the model must remember what it has learned even after many more updates. We call this setting long-horizon memorization. Prompting and retrieval can provide new information at inference time [Brown et al., 2020, Lewis et al., 2020], but the information remains outside the model’s parameters and must be supplied again. We instead ask whether repeated updates can build and preserve this memory within the model itself. We study this problem through continual supervised fine-tuning (SFT) in the domain-incremental setting [Van de Ven and Tolias, 2019]. Each task contains a set of query-answer pairs. The model learns 100 tasks in sequence without retaining raw examples from earlier tasks, and it receives no task identifier at inference. The goal is to learn each new task while retaining associations learned from previous tasks.

Discussion / Conclusion. Memorization rather than generalization. Our evaluation tests recall using the queries seen during training. A model may retain the corresponding associations yet answer a paraphrased query incorrectly. Our results therefore do not establish generalization to new query formulations. General capability preservation. Stronger memorization does not ensure preservation of general capability. The evaluated methods still lose substantial accuracy on general capability benchmarks after 100 tasks (Appendix E.9). Preserving these abilities while learning new associations remains an open challenge.