SYNTHESIS NOTE
Topics›Memory›this note

Why do retrieval systems fail on queries that never mention needed facts?

Memory retrieval assumes needed information will resemble the query text, but world knowledge often connects facts indirectly. This explores when that assumption breaks and what happens when it does.

Synthesis note · 2026-09-25 · sourced from Memory

The paper argues that retrieval-based memory rests on an assumption "so natural that it is rarely stated": a memory that is needed will resemble the query that needs it. World knowledge breaks that assumption. A tree-nut allergy should change the answer to a macaron request through the almond-flour ingredient, yet "the two texts share no cue a retriever can see." The authors call this the implicit-association blind spot and build InMind, a 125-task, expert-verified benchmark across ten life domains, to measure it. The headline numbers are stark: with the decisive memory placed in context, the backbone answers 84.0% of indirect queries, while six vector, graph, and agentic memory systems reach at most 14.4% when the same memory must be retrieved.

The mechanism the paper gives is structural rather than a matter of forgetting. Its introduction notes that an evaluated system recalled the allergy on demand seconds before recommending almond flour, and the abstract reports the systems recall the same facts on demand at up to 100%. The memory was stored and the model can reason from it, but it "was never brought to bear at the one moment it mattered." The paired controls are what license this reading, because they separate three explanations that existing evaluations conflate: the fact was never stored, the model lacks the bridging knowledge, or the fact was stored and never surfaced. The 84.0% in-context result rules out the second, and the on-demand recall rules out the first, which leaves surfacing. The one intervention the excerpt reports points the same way: an embedding with eight times the dimensionality raises "answer-blind target recall for every system yet leaves the gap essentially intact."

This locates a failure upstream of the one in Why do LLM agents remember preferences but not act on them?. That paper finds agents recalling a preference and then not using it, with comprehension errors dominating; here the model does use the memory when handed it, and the loss happens before the model sees anything. The two are compatible as different stages, but the excerpts do not reconcile them. It also gives a concrete failure mode for the decomposition argued in How should we actually evaluate agent memory systems?, where a recall-on-demand score would have passed these systems. And it qualifies Does retrieved memory quality depend on its functional role?: when relevance is judged by resemblance to the query, a memory can be essential and still look irrelevant.

The excerpt is short on what would make this actionable. It does not name the six systems or the backbone, say how indirect queries were built or scored beyond being expert-verified, break results out by domain, or define answer-blind target recall. It carries no discussion passage, so no remedy is proposed. What it supports is narrower and still useful: on this benchmark, recall on demand is not evidence that a memory will surface when a user's request never mentions it, and a larger embedding alone did not close the gap for the systems tested.

Inquiring lines that read this note 3

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Why does memory consolidation cause performance regression in continual learning? How should retrieval systems handle complex multi-step reasoning?

Related concepts in this collection 3

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
12 direct connections · 92 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

retrieval-based agent memory assumes a needed memory will resemble the query — the implicit-association blind spot breaks it on indirect queries