RePReL: Integrating Relational Planning and Reinforcement Learning for Effective Abstraction

Authors

  • Harsha Kokel The University of Texas at Dallas
  • Arjun Manoharan Robert Bosch Centre for Data Science and Artificial Intelligence at Indian Institute of Technology Madras
  • Sriraam Natarajan The University of Texas at Dallas
  • Balaraman Ravindran Robert Bosch Centre for Data Science and Artificial Intelligence at Indian Institute of Technology Madras
  • Prasad Tadepalli Oregon State University

Keywords:

Reinforcement Learning Using Planning (model-based, Bayesian, Deep, Etc.), Theoretical Aspects Of Planning And Learning, Applications That Involve A Combination Of Learning With Planning Or Scheduling, Learning To Improve The Effectiveness Of Planning & Scheduling Systems

Abstract

State abstraction is necessary for better task transfer in complex reinforcement learning environments. Inspired by the benefit of state abstraction in MAXQ and building upon hybrid planner-RL architectures, we propose RePReL, a hierarchical framework that leverages a relational planner to provide useful state abstractions. Our experiments demonstrate that the abstractions enable faster learning and efficient transfer across tasks. More importantly, our framework enables the application of standard RL approaches for learning in structured domains. The benefit of using the state abstractions is critical in relational settings, where the number and/or types of objects are not fixed apriori. Our experiments clearly show that RePReL framework not only achieves better performance and efficient learning on the task at hand but also demonstrates better generalization to unseen tasks.

Downloads

Published

2021-05-17

How to Cite

Kokel, H., Manoharan, A., Natarajan, S., Ravindran, B., & Tadepalli, P. (2021). RePReL: Integrating Relational Planning and Reinforcement Learning for Effective Abstraction. Proceedings of the International Conference on Automated Planning and Scheduling, 31(1), 533-541. Retrieved from https://ojs.aaai.org/index.php/ICAPS/article/view/16001