Should recommendation evaluation enforce probability competition between candidate items?
This explores whether recommenders should be trained and judged by making candidate items compete for a fixed probability budget — rather than scoring each item on its own — and what that competition does well and what it quietly distorts.
This explores whether recommenders should force candidate items to compete for a shared probability budget instead of scoring each item independently — and the corpus gives a surprisingly strong yes, with a catch worth knowing about. The core evidence comes from likelihood choice. When a model uses a multinomial likelihood, all items must split one fixed probability pool, so raising one item's score necessarily lowers another's. That competition turns out to match what recommendation actually is — picking the top few from many — which is why multinomial likelihoods beat Gaussian and logistic ones both for collaborative filtering Why does multinomial likelihood work better for ranking recommendations? and for click data Why does multinomial likelihood work better for click prediction?. The reason logistic and Gaussian losses underperform is precisely that they let many items be 'high probability' at once, which is comfortable for the model but misaligned with a ranking objective where only relative order matters.
So the competition framing isn't a tweak — it's the thing that makes training agree with the goal. But here's the part you didn't know you wanted to know: competition for a fixed budget also concentrates pressure, and that pressure has a direction. When embedding capacity is too small, the cheapest way to win the probability contest is to overfit toward popular items, which quietly produces long-term unfairness as niche items keep losing the competition and never get exposure Does embedding dimensionality secretly drive popularity bias in recommenders?. The same logic appears at the data layer: hash collisions don't fall evenly, they pile up on the high-frequency users and items that dominate the competition, degrading exactly the entities the model most needs to get right Why do hash collisions hurt recommendation models so much?. Probability competition optimizes ranking, but it also amplifies whatever is already winning.
That amplification is why enforcing competition in evaluation isn't automatically safe. A ranker that competes items against each other and trains on its own logged clicks can converge on a degenerate loop — it keeps recommending what it already recommended. YouTube's multi-objective work argues you have to explicitly model selection bias (with a position tower) and juggle conflicting objectives (with MMoE), or the competition just entrenches past decisions Why do ranking systems need to model selection bias explicitly?. So competition needs a counterweight: something that protects diversity rather than collapsing onto a single winner.
The corpus offers that counterweight from an unexpected angle. Instead of treating each user as one vector competing items head-to-head, representing a user as several weighted personas lets different candidate items win for different reasons — the model stays diverse and even explains which taste each recommendation satisfies, without a separate reranking step bolted on afterward Can attention mechanisms reveal which user taste explains each recommendation?. And opinion-dynamics work is a reminder that the competition you enforce shapes the world it measures: 'bought-together' versus 'co-viewed' recommendation structures push connected products' ratings to converge or diverge differently, so the competitive structure isn't a neutral scorecard — it feeds back into user behavior Do different recommender types shape opinion convergence differently?.
The synthesis: yes, enforce probability competition, because it's what aligns the objective with top-N ranking. But treat it as a powerful incentive with a popularity-amplifying bias baked in, and pair it with bias correction, adequate embedding capacity, and diversity-preserving structure — otherwise you'll have built a system that's excellent at ranking and quietly unfair over time.
Sources 7 notes
Liang et al. show that switching VAE likelihoods from Gaussian/logistic to multinomial achieves state-of-the-art results because enforced probability competition between items directly aligns training with top-N ranking objectives. Rebalancing KL regularization further improves performance.
Multinomial likelihood better models click data because it forces items to compete for a fixed probability budget, implicitly optimizing for top-N ranking. Gaussian and logistic likelihoods allow high probability across many items simultaneously, misaligning training with ranking objectives.
Research shows that when user/item embedding dimensions are too small, recommender systems overfit toward popular items to maximize ranking quality. This compounds over time as niche items receive insufficient exposure, and cannot be fixed post-hoc without treating dimensionality as a fairness hyperparameter.
Monolith's empirical work shows that real recommendation systems have power-law distributed frequencies, causing collisions to accumulate precisely on the entities models need most accurate. Fixed-size hashed tables worsen this over time as new IDs arrive.
YouTube's multi-objective ranker uses MMoE for conflicting objectives and a shallow position tower to remove selection bias from training data. Without both mechanisms, models converge on degenerate equilibria that amplify their own past decisions.
AMP-CF represents each user as multiple latent personas weighted dynamically by candidate item. This makes recommendations both diverse and interpretable—each suggestion traces to the specific persona preference it satisfies—without requiring post-hoc reranking.
Research shows that frequently-bought-together and co-viewed recommendation networks produce different opinion convergence patterns. The mechanism: each recommender type attracts different audience segments with different prior expectations, shaping both who sees products together and how they rate them.