How do token, parametric, and latent memory forms coexist in single agents?
This explores a 2025 survey's claim that agent memory comes in three physical forms — text the model reads (token), knowledge baked into weights (parametric), and information living in hidden activations (latent) — and asks how a single agent uses all three at once.
This explores how one agent juggles three different *places* memory can live: in the text it shells out to a context window (token), in the model's own weights (parametric), and in the hidden vectors flowing through the network (latent). The cleanest map for this is a recent survey that deliberately retires the old short-term/long-term split and replaces it with three axes — forms (token/parametric/latent), functions (factual/experiential/working), and dynamics (how memory forms, evolves, and gets retrieved) Can three axes replace the short-term long-term memory split?. The payoff of that reframing is that "short-term" and "long-term" stop being architectures and become *temporal patterns* — the same agent can hold a fact in token form for one turn and in parametric form forever, and the difference is duration, not a separate module.
In practice, today's agents lean overwhelmingly on the token form and route around the parametric one. A striking cluster of the corpus shows agents learning *without ever touching their weights*: Reflexion writes verbal self-diagnoses into episodic memory after each failure Can agents learn from failure without updating their weights?, and AgentFly formalizes the whole learning loop as memory operations over case, subtask, and tool stores — hitting 87.88% on GAIA with the base model frozen Can agents learn continuously from experience without updating weights?. The lesson is that what *could* be slow parametric updating gets deliberately pushed into fast, inspectable token memory. RAISE makes the token side even more granular, splitting working memory into dialogue-level and turn-level components that fail and update differently How should agent memory split across time scales?, while DeepAgent shows the token form doesn't have to grow forever — agents can autonomously *fold* their history into episodic, working, and tool schemas to fight context bloat Can agents compress their own memory without losing critical details?.
The latent form is the surprising frontier, and it's where coexistence stops being theoretical. Instead of serializing thoughts back into tokens, agents can pass raw hidden representations to each other: LatentMAS shares state directly through KV caches, cutting tokens 70–84% while *gaining* accuracy because the embeddings preserve reasoning that text would flatten latent-multi-agent-collaboration-achieves-training-free-lossless-information-exch. A companion line uses sparse autoencoders to pull individual, shared, and private "thoughts" out of those hidden states, even catching alignment conflicts before they ever surface as language Can agents share thoughts directly without using language?. So a single agent can be reading token memory, carrying latent memory in its activations, and only occasionally committing anything to parametric weights — three forms genuinely live side by side.
What ties this together is a structural argument the corpus makes repeatedly: reliability comes from *externalizing* cognition rather than scaling the model. One analysis names memory, skills, and protocols as three burdens that reliable agents offload into a harness layer instead of forcing the weights to re-solve them every run agent-reliability-comes-from-externalizing-cognitive-burdens-into-system-structures. Read against the three-forms taxonomy, that's exactly why token and latent memory are doing so much work while parametric memory stays mostly frozen — moving knowledge *out* of the weights is the whole design move.
The thing you might not have expected to want: the boundaries between these forms are porous and increasingly *chosen* per task. The non-linear-prompting work shows a single model can simulate a whole debate in token space and match what multi-agent systems do Can branching prompts replicate what multi-agent systems do?, and the SLM-sufficiency argument suggests the economical agent is heterogeneous by design — cheap small models for routine work, large ones selectively Can small language models handle most agent tasks?. Coexistence, then, isn't three modules bolted together; it's a continuous engineering decision about *where to put a given piece of knowledge* so it's cheap to write, faithful to retrieve, and durable for as long as it's needed.
Sources 10 notes
A 2025 survey reframes agent memory along forms (token/parametric/latent), functions (factual/experiential/working), and dynamics (formation/evolution/retrieval), showing that short/long-term phenomena emerge from temporal patterns rather than architectural separation. This enables precise system comparison and replaces vague implementation-based claims.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
RAISE shows that agent memory consists of four components organized by two design axes: dialogue-level (conversation history, scratchpad) versus turn-level (examples, task trajectory). This granularity distinction predicts different failure modes and update policies for each component.
DeepAgent's autonomous memory folding consolidates interaction history into episodic, working, and tool memory schemas. This reduces token overhead while letting agents pause to reconsider strategies—the autonomy and structure together avoid degradation that plagues poorly designed consolidation.
Research formalizes inter-agent thought sharing via sparse autoencoders that recover individual, shared, and private latent thoughts from hidden states. This approach detects alignment conflicts at the representational level before they manifest in language.
Research shows single LLMs using dynamic persona simulation achieve multi-agent cognitive synergy without multiple model instances. Solo Performance Prompting validates that structured prompting techniques map directly to multi-agent debate architectures, enabling equivalent outcomes through structural equivalence.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.