A User-Adaptive Layer Selection Framework for Very Deep Sequential Recommender Models

Authors

  • Lei Chen Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Fajie Yuan Tencent
  • Jiaxi Yang Huazhong University of Science and Technology
  • Xiang Ao Institute of Computing Technology, Chinese Academy of Sciences
  • Chengming Li Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
  • Min Yang Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences

Keywords:

Recommender Systems & Collaborative Filtering

Abstract

Sequential recommender systems (SRS) have become a research hotspot in recent studies. Because of the requirement in capturing user's dynamic interests, sequential neural network based recommender models often need to be stacked with more hidden layers (e.g., up to 100 layers) compared with standard collaborative filtering methods. However, the high network latency has become the main obstacle when deploying very deep recommender models into a production environment. In this paper, we argue that the typical prediction framework that treats all users equally during the inference phase is inefficient in running time, as well as sub-optimal in accuracy. To resolve such an issue, we present SkipRec, an adaptive inference framework by learning to skip inactive hidden layers on a per-user basis. Specifically, we devise a policy network to automatically determine which layers should be retained and which layers are allowed to be skipped, so as to achieve user-specific decisions. To derive the optimal skipping policy, we propose using gumbel softmax and reinforcement learning to solve the non-differentiable problem during backpropagation. We perform extensive experiments on three real-world recommendation datasets, and demonstrate that SkipRec attains comparable or better accuracy with much less inference time.

Downloads

Published

2021-05-18

How to Cite

Chen, L., Yuan, F., Yang, J., Ao, X., Li, C., & Yang, M. (2021). A User-Adaptive Layer Selection Framework for Very Deep Sequential Recommender Models. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5), 3984-3991. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16518

Issue

Section

AAAI Technical Track on Data Mining and Knowledge Management