Creativity of AI: Automatic Symbolic Option Discovery for Facilitating Deep Reinforcement Learning


  • Mu Jin Sun Yat-sen University
  • Zhihao Ma Sun Yat-Sen University
  • Kebing Jin Sun Yat-sen University
  • Hankz Hankui Zhuo Sun Yat-sen University
  • Chen Chen Huawei Noah’s Ark Lab
  • Chao Yu Sun Yat-sen University



Machine Learning (ML), Planning, Routing, And Scheduling (PRS)


Despite of achieving great success in real life, Deep Reinforcement Learning (DRL) is still suffering from three critical issues, which are data efficiency, lack of the interpretability and transferability. Recent research shows that embedding symbolic knowledge into DRL is promising in addressing those challenges. Inspired by this, we introduce a novel deep reinforcement learning framework with symbolic options. This framework features a loop training procedure, which enables guiding the improvement of policy by planning with action models and symbolic options learned from interactive trajectories automatically. The learned symbolic options help doing the dense requirement of expert domain knowledge and provide inherent interpretabiliy of policies. Moreover, the transferability and data efficiency can be further improved by planning with the action models. To validate the effectiveness of this framework, we conduct experiments on two domains, Montezuma's Revenge and Office World respectively, and the results demonstrate the comparable performance, improved data efficiency, interpretability and transferability.




How to Cite

Jin, M., Ma, Z., Jin, K., Zhuo, H. H., Chen, C., & Yu, C. (2022). Creativity of AI: Automatic Symbolic Option Discovery for Facilitating Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 36(6), 7042-7050.



AAAI Technical Track on Machine Learning I