Scalable Neural Contextual Bandit for Recommender Systems

High-quality recommender systems ought to deliver both innovative and relevant content through effective and exploratory interactions with users. Yet, supervised learning-based neural networks, which form the backbone of many existing recommender systems, only leverage recognized user interests, falling short when it comes to efficiently uncovering unknown user preferences. While there has been some progress with neural contextual bandit algorithms towards enabling online exploration through neural networks, their onerous computational demands hinder widespread adoption in real-world recommender systems. In this work, we propose a scalable sample-efficient neural contextual bandit algorithm for recommender systems. To do this, we design an epistemic neural network architecture, Epistemic Neural Recommendation (ENR), that enables Thompson sampling at a large scale. In two distinct large-scale experiments with real-world tasks, ENR significantly boosts clickthrough rates and user ratings by at least 9% and 6% respectively compared to state-of-the-art neural contextual bandit algorithms. Furthermore, it achieves equivalent performance with at least 29% fewer user interactions compared to the best-performing baseline algorithm.

Introduction. Recommender systems (RS), paramount in personalizing digital content, critically influence the quality of information accessed via the Internet. Traditionally, these systems have employed supervised learning algorithms, such as Collaborative Filtering [43], which have greatly benefited from advances in deep learning. These algorithms analyze vast quantities of data to discern user preferences; however, they are not designed to strategically probe in order to more quickly learn about user interests. Instead, they learn passively from collected data. Current research [23, 30, 47] reveals that deep-learning-driven RS tend to quickly confine their focus to a limited set of suboptimal topics, limiting their scope and hampering their learning capacity. This restrictive personalization strategy confines RS to recommend only those topics with which they have established familiarity, thus failing to discover and learn users’ other potential interests. The ability of an RS to identify and learn about user’s unexplored interests is a significant determinant of its long-term performance.

Discussion / Conclusion. In this paper, we designed a scalable and novel neural contextual bandit algorithm customized for recommender systems via a new epistemic neural network architecture and Thompson sampling. We formally define the recommender system problem as a contextual bandit problem and reviewed the current State-of-the-Art neural contextual bandit strategies. Our architecture design, Epistemic Neural Recommendation (ENR), presents much better scalability compared to other neural contextual bandit strategies. We show empirically through both synthetic experiments as well as two large-scale real-world experiments that ENR outperforms all other baselines. We hope that the results and the design in this paper inspires adoption of Epistemic Neural Recommendation as well as neural contextual bandit approaches in real-world systems.