Can simpler collaborative filtering models outperform deep architectures?
This explores whether shallow, linear collaborative-filtering models can beat deep neural recommenders — and the corpus says yes, surprisingly often, because the right structural constraint matters more than raw model capacity.
This explores whether simpler collaborative-filtering models can outperform deep architectures — and the recurring answer across the corpus is that they do, for a reason worth dwelling on: a well-chosen *constraint* beats added depth. The clearest case is EASE, a single linear item-item weight matrix whose only trick is forcing the diagonal to zero so an item can't predict itself. That one constraint pushes every prediction through genuine item-to-item relationships and lets the model learn negative weights that encode dissimilarity — and it beats deep autoencoders on most datasets Can simpler models beat deep networks for recommendation systems?. ESLER reaches the same conclusion from the same starting point: constrain self-similarity, let anti-affinity weights emerge, and a one-layer linear model outperforms most deep CF baselines Can a linear model beat deep collaborative filtering?. The headline isn't 'linear is better' — it's that the structural prior these models bake in is more valuable than the flexibility deep nets spend their capacity learning.
The same lesson shows up one level down, inside the deep models themselves. Rendle et al. found that a properly tuned dot product beats an MLP-based similarity function — even though an MLP is a universal approximator that *could* represent the dot product. The catch is that learning to reconstruct that simple geometric relationship takes enormous models and data, while the dot product gets it for free and runs efficiently at production scale Why does dot product beat MLP-based similarity in practice?. So 'can express it in principle' and 'will learn it in practice' are very different things — the inductive bias is doing the real work.
But the picture isn't simply 'simpler wins.' Sometimes the gain comes not from shrinking the model but from fixing the *objective*: swapping a VAE's Gaussian or logistic likelihood for a multinomial one forces items to compete for probability mass, which directly matches the top-N ranking task and sets a new state of the art — no extra depth required Why does multinomial likelihood work better for ranking recommendations?. That's the same family of insight as EASE: align the model's structure with the actual problem and capacity becomes secondary.
Where depth genuinely earns its keep is when there's something linear models structurally *can't* see. Graph autoencoders fold in side information to make predictions for brand-new users and items, cracking the cold-start problem that pure interaction-matrix methods can't touch Can autoencoders solve the cold-start problem in recommendations?. Attention-based persona models split a user into multiple latent tastes weighted by the candidate item, buying both accuracy and built-in explanations a single linear weight matrix won't give you Can attention mechanisms reveal which user taste explains each recommendation? Can modeling multiple user personas improve recommendation accuracy?.
So the honest synthesis is a question of *what you're buying*. If the job is plain item-to-item collaborative filtering on dense interaction data, the constrained linear model is often the strongest and cheapest option — the deep network's extra capacity is spent re-learning structure you could have just imposed. The moment you need cold-start handling, multiple distinct user tastes, or explainability, the deep architectures stop being overkill and start being the only thing that fits. The unintuitive takeaway: in recommendation, depth is not a default upgrade — it's a tool you reach for only when a constraint can't substitute for it.
Sources 7 notes
EASE, a shallow linear item-item weight matrix with diagonal constrained to zero, beats deep neural baselines on most datasets. The constraint forces generalization by forbidding self-prediction, while learned negative weights capture item dissimilarity—a structural prior more valuable than model capacity.
ESLER, a single-layer linear autoencoder constrained so items cannot predict themselves, outperforms most deep CF models. The constraint forces prediction through item relationships, and negative weights encoding anti-affinity prove essential—structural bias matters more than model capacity.
Rendle et al. show properly-tuned dot products substantially beat MLP-based similarity despite MLP universality. Learning a dot product with an MLP requires large models and datasets; dot products also enable efficient retrieval at production scale through MIPS algorithms.
Liang et al. show that switching VAE likelihoods from Gaussian/logistic to multinomial achieves state-of-the-art results because enforced probability competition between items directly aligns training with top-N ranking objectives. Rebalancing KL regularization further improves performance.
GHRS uses graph features and deep autoencoders to integrate rating history with side information, enabling predictions for new users and items by discovering non-linear relationships that linear hybrid methods miss.
AMP-CF represents each user as multiple latent personas weighted dynamically by candidate item. This makes recommendations both diverse and interpretable—each suggestion traces to the specific persona preference it satisfies—without requiring post-hoc reranking.
AMP-CF separates user representation into latent personas weighted by attention to the candidate item. This candidate-conditional approach improves accuracy by adapting the user representation at prediction time and produces inherent explanations for why items were recommended.