Tuning the Hyperparameters of Anytime Planning: A Metareasoning Approach with Deep Reinforcement Learning

Authors

  • Abhinav Bhatia University of Massachusetts Amherst
  • Justin Svegliato University of California, Berkeley
  • Samer B. Nashed University of Massachusetts Amherst
  • Shlomo Zilberstein University of Massachusetts Amherst

Keywords:

Metareasoning, Anytime Algorithms, Optimal Stopping, Hyperparameter Tuning, Anytime Weighted A*, Heuristic Search

Abstract

Anytime planning algorithms often have hyperparameters that can be tuned at runtime to optimize their performance. While work on metareasoning has focused on when to interrupt an anytime planner and act on the current plan, the scope of metareasoning can be expanded to tuning the hyperparameters of the anytime planner at runtime. This paper introduces a general, decision-theoretic metareasoning approach that optimizes both the stopping point and hyperparameters of anytime planning. We begin by proposing a generalization of the standard meta-level control problem for anytime algorithms. We then offer a meta-level control technique that monitors and controls an anytime algorithm using deep reinforcement learning. Finally, we show that our approach boosts performance on a common benchmark domain that uses anytime weighted A* to solve a range of heuristic search problems and a mobile robot application that uses RRT* to solve motion planning problems.

Downloads

Published

2022-06-13

How to Cite

Bhatia, A., Svegliato, J., Nashed, S. B., & Zilberstein, S. (2022). Tuning the Hyperparameters of Anytime Planning: A Metareasoning Approach with Deep Reinforcement Learning. Proceedings of the International Conference on Automated Planning and Scheduling, 32(1), 556-564. Retrieved from https://ojs.aaai.org/index.php/ICAPS/article/view/19842