Why do multinomial likelihoods outperform Gaussian models for recommendation?
This explores why a particular statistical choice — modeling user clicks as a multinomial 'pick one from a budget' rather than scoring each item independently — turns out to be the right fit for recommendation systems whose real job is ranking.
This explores why multinomial likelihoods beat Gaussian (and logistic) ones in recommenders, and the answer turns out to be less about statistics than about what you're secretly optimizing for. The core insight is competition: a multinomial likelihood forces all items to share a single fixed probability budget, so raising the score of one item means lowering others. Gaussian and logistic likelihoods have no such constraint — they happily assign high probability to many items at once. That sounds harmless until you remember that recommendation isn't really a prediction task, it's a ranking task. You only get to show a user the top handful of items, so a loss function that makes items compete for limited probability is implicitly training the model on the thing you actually care about: top-N ranking Why does multinomial likelihood work better for click prediction?. Liang et al. made this concrete by swapping the likelihood inside a variational autoencoder and watching collaborative-filtering results jump to state-of-the-art — with a further boost from rebalancing the KL regularization Why does multinomial likelihood work better for ranking recommendations?.
The deeper lesson is a recurring theme in this corpus: in recommendation, getting the *objective* aligned with ranking matters more than raw model power. The multinomial story is one instance of a pattern where a structural constraint baked into the model does the heavy lifting. ESLER is the cleanest cousin — a single linear layer that beats most deep models simply by forbidding items from predicting themselves, forcing every prediction to flow through item-to-item relationships. The takeaway there is explicit: structural bias matters more than model capacity Can a linear model beat deep collaborative filtering?. Multinomial likelihoods and zero-diagonal constraints are doing the same kind of work from different angles — shaping what the model is allowed to do so that learning lines up with ranking.
The flip side shows up when you optimize ranking quality too aggressively without watching what it costs. When embedding dimensions are too small, models overfit toward popular items precisely *because* that maximizes ranking metrics — and that compounds into long-term unfairness as niche items never get exposure Does embedding dimensionality secretly drive popularity bias in recommenders?. So the same competitive, ranking-aligned pressure that makes multinomial likelihoods effective can, under the wrong capacity constraints, quietly concentrate recommendations on the already-popular. The probability budget has to come from somewhere.
Worth knowing for the curious: the field is now skipping the likelihood-engineering question altogether in some setups by training directly on the ranking metrics themselves. Rec-R1 uses NDCG and Recall as reinforcement-learning reward signals to train language models, treating recommendation quality as a black-box reward rather than something to approximate through a clever loss function Can recommendation metrics train language models directly?. That's the logical endpoint of the multinomial insight — if the whole point of choosing multinomial was to align training with top-N ranking, why not optimize the ranking metric directly?
Sources 5 notes
Multinomial likelihood better models click data because it forces items to compete for a fixed probability budget, implicitly optimizing for top-N ranking. Gaussian and logistic likelihoods allow high probability across many items simultaneously, misaligning training with ranking objectives.
Liang et al. show that switching VAE likelihoods from Gaussian/logistic to multinomial achieves state-of-the-art results because enforced probability competition between items directly aligns training with top-N ranking objectives. Rebalancing KL regularization further improves performance.
ESLER, a single-layer linear autoencoder constrained so items cannot predict themselves, outperforms most deep CF models. The constraint forces prediction through item relationships, and negative weights encoding anti-affinity prove essential—structural bias matters more than model capacity.
Research shows that when user/item embedding dimensions are too small, recommender systems overfit toward popular items to maximize ranking quality. This compounds over time as niche items receive insufficient exposure, and cannot be fixed post-hoc without treating dimensionality as a fairness hyperparameter.
Rec-R1 demonstrates that LLMs can be trained directly on rule-based recommendation metrics like NDCG and Recall as RL reward signals, eliminating the need for SFT distillation from proprietary models while remaining model-agnostic across different retriever architectures.