Hierarchical Reinforcement Learning for Integrated Recommendation

Authors

  • Ruobing Xie WeChat Search Application Department, Tencent
  • Shaoliang Zhang WeChat Search Application Department, Tencent
  • Rui Wang WeChat Search Application Department, Tencent
  • Feng Xia WeChat Search Application Department, Tencent
  • Leyu Lin WeChat Search Application Department, Tencent

Keywords:

Recommender Systems & Collaborative Filtering, Reinforcement Learning

Abstract

Integrated recommendation aims to jointly recommend heterogeneous items in the main feed from different sources via multiple channels, which needs to capture user preferences on both item and channel levels. It has been widely used in practical systems by billions of users, while few works concentrate on the integrated recommendation systematically. In this work, we propose a novel Hierarchical reinforcement learning framework for integrated recommendation (HRL-Rec), which divides the integrated recommendation into two tasks to recommend channels and items sequentially. The low-level agent is a channel selector, which generates a personalized channel list. The high-level agent is an item recommender, which recommends specific items from heterogeneous channels under the channel constraints. We design various rewards for both recommendation accuracy and diversity, and propose four losses for fast and stable model convergence. We also conduct an online exploration for sufficient training. In experiments, we conduct extensive offline and online experiments on a billion-level real-world dataset to show the effectiveness of HRL-Rec. HRL-Rec has also been deployed on WeChat Top Stories, affecting millions of users. The source codes are released in https://github.com/modriczhang/HRL-Rec.

Downloads

Published

2021-05-18

How to Cite

Xie, R., Zhang, S., Wang, R., Xia, F., & Lin, L. (2021). Hierarchical Reinforcement Learning for Integrated Recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5), 4521-4528. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16580

Issue

Section

AAAI Technical Track on Data Mining and Knowledge Management