Search tools find text that sounds like your question — but what if the fact you need shares no words with it?
Can relevance judgments based on query resemblance miss essential factual connections?
This explores whether retrieval systems that rank passages by how much they resemble the question can fail to find the facts an answer depends on, and what the corpus says about why and how to fix it.
This explores whether retrieval systems that rank passages by how much they resemble the question can fail to find the facts an answer depends on. The corpus says yes, and the failure is large. InMind, a memory benchmark, gives models indirect requests whose answers depend on a fact the request never mentions. With the right memory placed in context, models answer 84% correctly. When the same memory has to be retrieved, they answer only 14.4% correctly. The system had stored the fact, but the question shared no words with it, so it never came up (Why do retrieval systems fail on queries that never mention needed facts?).
The reason is that being about the same topic and being what explains the question are different things. In backtracing, a student asks about projection right after a lecturer says something specific. The passage closest in meaning discusses projection matrices, but the real cause of the confusion is the earlier statement that never mentions the word (Why do queries and their causes seem semantically different?). The library's overview of retrieval failures makes the same point at a broader level. Embeddings measure association rather than relevance, and the dimension of an embedding puts a hard mathematical cap on which sets of documents it can represent. The overview treats these as architectural limits that tuning won't fix (Where do retrieval systems fail and why?).
Resemblance also fails in the other direction. It can pull in things that look right and aren't. GrepSeek's authors found that dense embeddings blur similar entities together, so an entity-constrained multi-hop question gets passages about the wrong entity. Their agent issues plain text-search commands over the raw corpus and recovers the exact-match precision that embeddings lose (Can direct corpus search beat embedding-based retrieval?). A separate line of work treats "same topic, wrong identity" as its own problem. It adds a small verifier after the initial recall step that looks at token-by-token similarity patterns and rejects structural near-misses that standard late-interaction scoring accepts (Can verification separate structural near-misses from topical matches?).
The fixes come from quite different directions, and each one changes what counts as a match. One adds a missing dimension of relevance, such as time. TempRALM scores documents on temporal fit as well as semantic similarity, and gains up to 74% when a document has several time-stamped versions, with no retraining (Can retrieval systems ground answers in the right time?). Another lets the question type decide the retrieval strategy. StructRAG routes each query to the knowledge structure that suits it, such as a table, graph or catalogue, and non-factoid questions split into five types that each need different retrieval (Can routing queries to task-matched structures improve RAG reasoning?, Does question type determine the right retrieval strategy?). A third splits query planning from answer synthesis, which helps on multi-hop questions where the needed fact is a step away from the question (Do hierarchical retrieval architectures outperform flat ones on complex queries?). The catch is that you can't just tell the retriever what you mean. Most retrievers ignore natural-language instructions about what counts as relevant, and only models above roughly 3B parameters or those trained on instructions respond to them (Do retrieval models actually follow natural language instructions?).
Getting the right memory back doesn't finish the job. MemTrapBench found that all five memory frameworks it tested did worse than using no memory at all, by more than 10%, even when the stored memories were accurate and on topic (Can relevant memories actually harm LLM reasoning?). Resemblance-based relevance can miss the fact you need, and a retrieved fact can still hurt once the model has it.
Sources 11 notes
InMind's benchmark shows that with memory in context, models answer 84% of indirect queries correctly, but retrieval systems answer only 14.4% when the same memory must be retrieved. The systems store and recall the facts on demand, yet fail to surface them when user requests never mention connecting terms.
Backtracing—finding what caused a query—diverges from semantic similarity especially in conversation and lecture domains. Students ask about projection after hearing a specific statement, but the semantically closest passage discusses projection matrices instead, showing that surface similarity misses the actual cause.
RAG systems fail at three structural levels: adaptive triggering (fixed intervals waste context), semantic-task mismatch (embeddings measure association, not relevance), and mathematical limits (embedding dimension constrains representable document sets). These require fundamentally different retrieval approaches, not tuning.
GrepSeek trains agents to retrieve via executable shell commands over raw text, achieving better multi-hop performance on entity-constrained queries than dense embeddings. The approach scaffolds unstable search mechanics with supervised trajectories, then refines task-oriented behavior through reinforcement learning.
A two-stage pipeline—pooled-cosine recall followed by a small Transformer verifier operating on token-token similarity maps—reliably rejects structural near-misses that MaxSim-style late interaction cannot. The verifier succeeds because it operates on full token interaction patterns rather than compressed vectors.
Show all 11 sources
TempRALM adds a temporal term to retrieval scoring alongside semantic similarity, achieving up to 74% improvement over baseline systems when documents have multiple time-stamped versions. The approach requires no model retraining or index changes.
StructRAG demonstrates that selecting knowledge structure type based on query demands—via DPO-trained router choosing among tables, graphs, algorithms, catalogues, and chunks—improves knowledge-intensive reasoning over standard retrieval. The approach grounds this in cognitive load and cognitive fit theory from cognitive science.
Research shows non-factoid questions split into five types, each requiring different retrieval and aggregation methods. Evidence-based questions suit standard RAG, while debate and comparison need aspect-specific retrieval, and experience/reason questions need decomposition or filtering strategies.
Separating query planning from answer synthesis into distinct components reduces interference and improves multi-hop query performance. This architectural principle mirrors documented benefits of separating planning from execution in agent design.
A benchmark built from TREC narratives shows nearly all retrievers fail to adjust relevance decisions based on natural language instructions. Only models with 3B+ parameters or instruction-tuning learn to follow them, though training can teach this capability.
MemTrapBench shows that all five tested memory frameworks underperform a no-memory baseline, with drops exceeding 10%, despite memories being accurately stored and task-relevant. This reveals a failure mode at the point of use that standard memory benchmarks miss.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- On the Theoretical Limitations of Embedding-Based Retrieval
- Chain-of-Retrieval Augmented Generation
- Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
- A New Role for Relevance: Guiding Corpus Interaction in Agentic Search
- You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
- FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
- Backtracing: Retrieving the Cause of the Query
- UR2: Unify RAG and Reasoning through Reinforcement Learning