Should agent memory live inside the model backbone?

The Metis paper starts from an asymmetry: agent capabilities such as multimodal perception and reasoning have been "internalized" into foundation models, yet "agent memory is still primarily implemented through external modules." It proposes memory foundation models to close that gap and defines native memory in two parts. The first is "a persistent and dynamically evolving memory state within the backbone." The second is "native memory procedures that autonomously store and utilize information through model computation." The paper says this offers advantages in architecture, end-to-end optimization, and efficiency, and presents Metis as "the first prototype."

The reasoning in the introduction is a decoupling argument. External memory is "decoupled from backbones with separated targets and processing stages": the memory system builds an informative context, and the backbone then performs conditional language modeling over whatever it was handed. Two failures follow. The memory may not supply what the backbone needs, and the backbone may not use the memory optimally, because neither side is trained against the other's behavior. Metis answers with a different architecture. "Metis blocks" combine local memory blocks and hyper memory blocks, keeping "compact dense memory states across interaction steps" and updating them from the current input and the generated response. History is compressed into the model and read through memory attention. The store-and-use procedures are acquired in mid-training, using a memory-specific dataset built from public benchmarks and three kinds of objective (memory reconstruction, memory operation, regularization).

This sits close to Can neural memory modules scale language models beyond attention limits?, which also puts a compressed, persistent memory inside the network and reads it alongside attention. The difference in emphasis is that Metis treats memory as a capability class the model acquires through training, comparable to reasoning, and not only as an architectural component for long context. It also reframes what How should we actually evaluate agent memory systems? describes. In an external system those four stages are separable modules that can be inspected. In a native design they fold into model computation, so the decomposition that note recommends has no obvious place to attach. The failure modes in Do memory systems actually help language models learn continuously? and Does agent memory degrade when continuously consolidated? belong to external, text-consolidating memory. Native memory is a candidate response to them, but only in the sense that it removes the hand-off the introduction criticizes.

The excerpt does not establish the payoff. It says experiments "verify the effectiveness of Metis" and analyze its behavior, but it gives no benchmarks, baselines, model sizes, memory capacity, or numbers. It also says nothing about how a compressed memory state handles stale or wrong content, and it calls Metis "an early step." The introduction passage shows only the first of the "several limitations" of external memory that its Figure 1 lists. What the paper supports at this strength is a formulation and a prototype design: the decoupling between memory and backbone is a specific, nameable cost of external memory, and end-to-end training is the proposed remedy. Whether that remedy beats external memory on the tasks where external memory fails is left to the full paper and to later work.