Unsupervised Cross-Domain Transfer in Policy Gradient Reinforcement Learning via Manifold Alignment


  • Haitham Bou Ammar University of Pennsylvania
  • Eric Eaton University of Pennsylvania
  • Paul Ruvolo Olin College of Engineering
  • Matthew Taylor Washington State University




transfer learning, reinforcement learning, policy gradients, manifold alignment, cross-domain transfer


The success of applying policy gradient reinforcement learning (RL) to difficult control tasks hinges crucially on the ability to determine a sensible initialization for the policy. Transfer learning methods tackle this problem by reusing knowledge gleaned from solving other related tasks. In the case of multiple task domains, these algorithms require an inter-task mapping to facilitate knowledge transfer across domains. However, there are currently no general methods to learn an inter-task mapping without requiring either background knowledge that is not typically present in RL settings, or an expensive analysis of an exponential number of inter-task mappings in the size of the state and action spaces. This paper introduces an autonomous framework that uses unsupervised manifold alignment to learn inter-task mappings and effectively transfer samples between different task domains. Empirical results on diverse dynamical systems, including an application to quadrotor control, demonstrate its effectiveness for cross-domain transfer in the context of policy gradient RL.




How to Cite

Bou Ammar, H., Eaton, E., Ruvolo, P., & Taylor, M. (2015). Unsupervised Cross-Domain Transfer in Policy Gradient Reinforcement Learning via Manifold Alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). https://doi.org/10.1609/aaai.v29i1.9631



Main Track: Novel Machine Learning Algorithms