Why does the hot-path cold-path split map onto formation and evolution?
This explores why the engineering distinction between real-time work ('hot path') and deferred background work ('cold path') lines up with two of the memory dynamics — forming new memories vs. evolving old ones — in recent agent-memory thinking.
This reads the question as: when an AI agent manages memory, why do the operations that have to happen live, in the moment, turn out to be the *formation* of memory, while the operations you can push to the background turn out to be its *evolution*? The clearest map for this comes from a 2025 survey that reframes agent memory along three dynamics — formation, evolution, and retrieval — arguing these are more fundamental than the familiar short-term/long-term split Can three axes replace the short-term long-term memory split?. The hot-path/cold-path divide is really a timing question laid over those dynamics: formation and retrieval are *coupled to the live interaction* (you can't retrieve a fact after the user has left, and you can't form a memory of a turn that hasn't happened), so they sit on the hot path. Evolution — consolidation, reorganization, pruning, merging — has no such deadline. It can be batched, deferred, and amortized, so it falls naturally onto the cold path.
The deeper reason the split holds is about what each operation *costs* and *when the cost pays off*. Formation is cheap-per-event but unavoidably synchronous — it rides along with whatever the agent is already doing. Evolution is expensive and global: it wants to look across many memories at once and restructure them, which is exactly the kind of work you don't want blocking a response. The Memory-Amortized Inference view sharpens this: intelligence as the *reuse* of structured prior inference paths, where the heavy lifting of organizing memory into a navigable topology is an investment made once (cold) and then cheaply traversed many times (hot) Can cognition work by reusing memory instead of recomputing?. Evolution is that investment; formation and retrieval are the navigation that cashes it in.
There's a useful contrast in the corpus from systems that deliberately *refuse* to keep an evolving memory. Atom of Thoughts contracts reasoning into a memoryless Markov chain where each state depends only on the current problem, not the accumulated history Can reasoning systems forget history without losing coherence?. That's a design that collapses the cold path entirely — no evolution, only formation-and-discard — and it works precisely because reasoning baggage is the thing it wants to shed. Reading it against the three-axes survey tells you the hot/cold split isn't a law of nature; it's a choice about *whether evolution earns its keep*. When stored memory is a liability rather than an asset, you keep only the hot path.
The thing you may not have known you wanted to know: the same hot/cold logic shows up in inference-time *search*, not just memory. Evolutionary search at inference time maintains a diverse population that improves over many rounds — a slow, background, exploration-heavy process — while any single answer is generated fast and forward Can evolutionary search beat sampling and revision at inference time?. 'Formation' (produce a candidate now) and 'evolution' (improve the population over time) recur as the same two-speed pattern wherever a system has to act immediately but also get better slowly. The hot-path/cold-path split maps onto formation and evolution because it's the general shape of any agent that must both *respond* and *learn from responding* — and those two have fundamentally different clocks.
Worth being straight: only one note in this corpus directly theorizes the formation/evolution/retrieval framing, so the literal mapping rests largely on it; the rest are lateral reads that show the same two-speed structure recurring elsewhere rather than confirming the survey's taxonomy.
Sources 4 notes
A 2025 survey reframes agent memory along forms (token/parametric/latent), functions (factual/experiential/working), and dynamics (formation/evolution/retrieval), showing that short/long-term phenomena emerge from temporal patterns rather than architectural separation. This enables precise system comparison and replaces vague implementation-based claims.
Memory-Amortized Inference proposes intelligence arises from structured reuse of prior inference paths over topological memory, inverting RL's reward-forward logic into cause-backward reconstruction. This duality explains energy efficiency and suggests memory trajectories form the substrate of adaptive thought.
Atom of Thoughts decomposes problems into DAGs and contracts them iteratively, ensuring each state depends only on the current problem—not prior steps. This memoryless approach eliminates historical baggage that bloats reasoning while maintaining answer equivalence.
Mind Evolution uses genetic algorithms with LLM-generated mutations and crossovers to significantly outperform Best-of-N and Sequential Revision on planning benchmarks. An island model sustains population diversity, preventing the premature convergence that single-trajectory refinement exhibits.