Enhancing Context-Based Meta-Reinforcement Learning Algorithms via An Efficient Task Encoder (Student Abstract)

Authors

  • Feng Xu National Key Laboratory for Novel Software Technology, Nanjing University
  • Shengyi Jiang National Key Laboratory for Novel Software Technology, Nanjing University
  • Hao Yin National Key Laboratory for Novel Software Technology, Nanjing University
  • Zongzhang Zhang National Key Laboratory for Novel Software Technology, Nanjing University
  • Yang Yu National Key Laboratory for Novel Software Technology, Nanjing University
  • Ming Li National Key Laboratory for Novel Software Technology, Nanjing University
  • Dong Li Noah’s Ark Lab, Huawei Company
  • Wulong Liu Noah’s Ark Lab, Huawei Company

Keywords:

Meta Learning, Reinforcement Learning, Representation Learning

Abstract

Meta-Reinforcement Learning (meta-RL) algorithms enable agents to adapt to new tasks from small amounts of exploration, based on the experience of similar tasks. Recent studies have pointed out that a good representation of a task is key to the success of off-policy context-based meta-RL. Inspired by contrastive methods in unsupervised representation learning, we propose a new method to learn the task representation based on the mutual information between transition tuples in a trajectory and the task embedding. We also propose a new estimation for task similarity based on Q-function, which can be used to form a constraint on the distribution of the encoded task variables, making the task encoder encode the task variables more effective on new tasks. Experiments on meta-RL tasks show that the newly proposed method outperforms existing meta-RL algorithms.

Downloads

Published

2021-05-18

How to Cite

Xu, F., Jiang, S., Yin, H., Zhang, Z., Yu, Y., Li, M., Li, D., & Liu, W. (2021). Enhancing Context-Based Meta-Reinforcement Learning Algorithms via An Efficient Task Encoder (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15937-15938. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17965

Issue

Section

AAAI Student Abstract and Poster Program