What mathematical limits constrain embedding-based retrieval systems?

This explores the hard ceilings on embedding retrieval: the failures that are mathematical, not matters of better tuning. The corpus is unusually direct here — the central finding is that embedding dimension itself caps what a retriever can do. Drawing on communication complexity theory, researchers prove that for any embedding dimension d, there's a maximum number of top-k document combinations the system can ever return — and crucially, this limit holds even when embeddings are optimized directly on the test data, and even on trivially simple tasks Do embedding dimensions fundamentally limit retrievable document combinations?. You cannot represent arbitrarily many distinct 'which documents go together' answers in a fixed-dimensional vector space. More dimensions push the ceiling higher, but the ceiling never disappears.

That dimensional limit is one of three structural failure points the corpus identifies, and seeing all three together reframes the issue: RAG doesn't fail incrementally, it fails architecturally Where do retrieval systems fail and why?. Alongside the dimension cap sit adaptive triggering (fixed retrieval intervals waste context) and a subtler semantic problem — embeddings measure association, not relevance. This second limit is conceptual rather than information-theoretic but just as load-bearing: vectors encode co-occurrence, so concepts that are semantically close but play opposite roles look nearly identical to the system Do vector embeddings actually measure task relevance?. It demos beautifully and breaks in production, where underspecified queries surface many wrong-but-associated candidates. The dimension limit says 'you can't return every combination'; the association limit says 'the similarities you can return may be measuring the wrong thing.'

What's quietly interesting is that the same embedding geometry which constrains retrieval also carries real structure. The leading eigenvectors of an embedding Gram matrix split a taxonomy coarse-to-fine, tracking the WordNet hypernym tree level by level Do embedding eigenvectors organize taxonomy from coarse to fine?, and LLM activations encode syntactic type and direction in polar coordinates, not just distance How do language models encode syntactic relations geometrically?. So the limit isn't that embeddings are dumb — it's that a single similarity score collapses rich structure into one number, and one number can only separate so much.

The corpus's most useful move is showing the escape routes, which all share a shape: stop asking a single embedding-similarity step to carry the whole load. Describe an image in natural language and retrieve over text instead of raw visual embeddings Can describing images in text improve zero-shot recognition?. Discretize text into codes so representations decouple from the text encoder Can discretizing text embeddings improve recommendation transfer?. Separate query planning from answer synthesis into distinct stages Do hierarchical retrieval architectures outperform flat ones on complex queries?. Or sidestep retrieval entirely with long-context models — though these match RAG only on semantic tasks and collapse on structured relational queries that need joins Can long-context LLMs replace retrieval-augmented generation systems?. Even the decision of *when* to retrieve can beat the embedding bottleneck: calibrated uncertainty estimates outperform heuristic adaptive retrieval at a fraction of the cost Can simple uncertainty estimates beat complex adaptive retrieval?.

The thing you didn't know you wanted to know: the limits aren't a reason embedding retrieval is broken — they're a map of where to add a second mechanism. Single-vector similarity is a compression, and the research direction the corpus points toward is decomposition — language descriptions, discrete codes, planning/synthesis splits — that route around what one fixed-dimensional dot product can't express.