Why does AI search treat a product and its rival as the same thing, just because they sound alike?
Why do vector embeddings fail when semantically similar entities should be treated as distinct?
This explores why embedding-based search and matching treats two things as interchangeable when they are close in meaning but a task needs them kept apart, such as a product and its competitor, or a near-duplicate that isn't the same item.
This explores why embedding-based search and matching treats two things as interchangeable when they are close in meaning but a task needs them kept apart, such as a product and its competitor, or a near-duplicate that isn't the same item. The corpus's core answer is that embeddings measure association, not role. They are built from co-occurrence patterns, so concepts that appear in the same contexts land close together even when they play different parts in your question Do vector embeddings actually measure task relevance?. In a clean demo the right answer is the nearest neighbor. In production, an underspecified query has many wrong-but-associated candidates, and they all score high.
This isn't because embeddings are empty. Even static embeddings carry rich meaning such as valence and concreteness Do transformer static embeddings actually encode semantic meaning?. The problem is that the meaning they capture is not the distinction you need. There is also a coarse-to-fine pattern: the leading directions in embedding space split broad taxonomy branches first and finer sub-branches later, mirroring WordNet Do embedding eigenvectors organize taxonomy from coarse to fine?. A plausible consequence is that the loud signal is 'these are both animals', while 'but not the same animal' is a quieter one that a single similarity score can drown out.
The score itself is also less solid than it looks. Cosine similarities between learned embeddings can depend on regularization choices made in training rather than on real semantic structure, which makes them potentially arbitrary Does cosine similarity actually measure embedding similarity?. The geometry has a hard ceiling too: for any embedding dimension, some combinations of top-k results simply cannot be returned, even on trivially simple tasks Do embedding dimensions fundamentally limit retrievable document combinations?. Recommendation systems hit the same wall from another side. One fixed-length user vector squeezes many different interests together, and the fix there is to weigh the user's history against each specific candidate instead of comparing one compressed summary How can user vectors capture diverse interests without exploding in size?. The pattern is that compressing first and comparing later loses the distinctions you needed.
The fixes all stop asking one vector to settle identity. A two-stage pipeline uses cheap cosine recall to gather candidates, then a small verifier looks at the full token-to-token similarity map. It reliably rejects structural near-misses that late-interaction scoring (MaxSim) cannot Can verification separate structural near-misses from topical matches?. For relational questions, graph databases replace probabilistic similarity with deterministic traversal, at the price of higher construction cost When do graph databases outperform vector embeddings for retrieval?. In recommendation, discretizing text into codes stops text similarity from dictating what counts as alike Can discretizing text embeddings improve recommendation transfer?. The library frames these as architectural failures rather than tuning problems Where do retrieval systems fail and why?. Similarity finds what is nearby, and deciding whether two nearby things are the same is a separate job.
Sources 10 notes
Embeddings encode co-occurrence patterns, making semantically close but role-distinct concepts highly similar. This works in simple demos but fails in production where underspecified queries have many wrong-but-associated candidates.
Clustering analysis of RoBERTa embeddings reveals sensitivity to five psycholinguistic measures including valence, concreteness, iconicity, and taboo. This demonstrates that static embeddings function as genuine lexical entries containing semantic content before self-attention operates.
Leading eigenvectors of embedding Gram matrices separate broad taxonomic branches first, then progressively finer sub-branches—a coarse-to-fine spectral order that tracks the WordNet hypernym tree level by level, confirming predictions from co-occurrence statistics.
Regularized linear models with closed-form solutions show that cosine similarities between embeddings are not unique and depend on regularization choices made during training, not on actual semantic structure. This makes cosine scores unstable and potentially meaningless.
Communication complexity theory proves that for any embedding dimension d, there exists a maximum number of top-k document combinations that can be returned as results. Even embeddings optimized directly on test data hit this polynomial limit, demonstrated on trivially simple retrieval tasks.
Show all 10 sources
Deep Interest Network weights historical behaviors against each candidate ad, activating only relevant interests dynamically. This preserves dimension efficiency while expressing diverse tastes without lossy compression.
A two-stage pipeline—pooled-cosine recall followed by a small Transformer verifier operating on token-token similarity maps—reliably rejects structural near-misses that MaxSim-style late interaction cannot. The verifier succeeds because it operates on full token interaction patterns rather than compressed vectors.
Graph-oriented databases solve vector similarity's failure on aggregate queries by replacing probabilistic similarity search with deterministic graph traversal via Cypher. The tradeoff: higher construction cost but precision and completeness for enterprise use cases where query patterns are relational.
VQ-Rec uses product quantization to map item text to discrete codes that index learned embeddings, breaking the tight coupling between text and recommendations. This decoupling prevents text-similarity bias and allows lookup tables to adapt to new domains without retraining the text encoder.
RAG systems fail at three structural levels: adaptive triggering (fixed intervals waste context), semantic-task mismatch (embeddings measure association, not relevance), and mathematical limits (embedding dimension constrains representable document sets). These require fundamentally different retrieval approaches, not tuning.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- On the Theoretical Limitations of Embedding-Based Retrieval
- Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words
- Semantic Structure in Large Language Model Embeddings
- Training for Compositional Sensitivity Reduces Dense Retrieval Generalization
- Topic Modeling in Embedding Spaces
- Retrieval-augmented reasoning with lean language models
- Is Cosine-Similarity of Embeddings Really About Similarity?
- Word Meanings in Transformer Language Models