Can a separate memory model inject knowledge without touching the LLM?

The standard taxonomy of getting fresh knowledge into a frozen LLM offers three doors: non-parametric retrieval (RAG), parametric fine-tuning, and latent memory. MeMo proposes a fourth — encode the new corpus into a separate, trainable MEMORY model that the frozen LLM queries, with the base LLM untouched. This is not retrieval (no datastore search at inference) and not fine-tuning (the LLM's weights never move), which lets it sidestep the failure modes of each: it captures cross-document relationships RAG's chunk-level retrieval misses, it avoids the catastrophic forgetting parametric updates risk, and because it needs neither weights nor output logits it stays plug-and-play with closed proprietary models.

The load-bearing property is that inference-time retrieval cost is independent of corpus size. RAG's nearest-neighbor search and context budget scale with the store; MeMo amortizes the corpus into the memory model up front, then decomposes complex queries into targeted sub-queries at inference. This places it squarely alongside Can retrieval knowledge compress into a tiny parametric model? — both replace search with a trained decoder — and it extends How do knowledge injection methods trade off flexibility and cost? by populating that taxonomy's high-training-cost / fast-inference quadrant. It is also the document-knowledge analogue of Can a single model replace retrieval for long-term conversation memory?, which makes the same move for conversation rather than corpora.

The honest cost is that the bottleneck is relocated, not removed. The memory model must be trained on a synthesized reflection-QA dataset, so the price is training cost and — the paper concedes — the memory model's own capacity to scale with corpus size. Therefore MeMo is not a universal win over RAG: it trades RAG's per-query latency and weak cross-document reasoning for an up-front training burden and a capacity ceiling, which means it pays off precisely when a corpus is queried many times and demands multi-hop reasoning, and loses when the corpus is large, volatile, or rarely hit.