Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management

Authors

  • Rundong Wang Nanyang Technological University
  • Hongxin Wei Nanyang Technological University
  • Bo An Nanyang Technological University
  • Zhouyan Feng WeBank Co. Ltd.
  • Jun Yao WeBank Co. Ltd.

Keywords:

Economic/Financial, Reinforcement Learning

Abstract

Portfolio management via reinforcement learning is at the forefront of fintech research, which explores how to optimally reallocate a fund into different financial assets over the long term by trial-and-error. Existing methods are impractical since they usually assume each reallocation can be finished immediately and thus ignoring the price slippage as part of the trading cost. To address these issues, we propose a hierarchical reinforced stock trading system for portfolio management (HRPM). Concretely, we decompose the trading process into a hierarchy of portfolio management over trade execution and train the corresponding policies. The high-level policy gives portfolio weights at a lower frequency to maximize the long-term profit and invokes the low-level policy to sell or buy the corresponding shares within a short time window at a higher frequency to minimize the trading cost. We train two levels of policies via a pre-training scheme and an iterative training scheme for data efficiency. Extensive experimental results in the U.S. market and the China market demonstrate that HRPM achieves significant improvement against many state-of-the-art approaches.

Downloads

Published

2021-05-18

How to Cite

Wang, R., Wei, H., An, B., Feng, Z., & Yao, J. (2021). Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management. Proceedings of the AAAI Conference on Artificial Intelligence, 35(1), 626-633. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16142

Issue

Section

AAAI Technical Track on Application Domains