Stop! Planner Time: Metareasoning for Probabilistic Planning Using Learned Performance Profiles
DOI:
https://doi.org/10.1609/aaai.v38i18.29983Keywords:
PRS: Learning for Planning and Scheduling, PRS: Planning under Uncertainty, PRS: Planning with Markov Models (MDPs, POMDPs), RU: Sequential Decision Making, SO: Metareasoning and MetaheuristicsAbstract
The metareasoning framework aims to enable autonomous agents to factor in planning costs when making decisions. In this work, we develop the first non-myopic metareasoning algorithm for planning with Markov decision processes. Our method learns the behaviour of anytime probabilistic planning algorithms from performance data. Specifically, we propose a novel model for metareasoning, based on contextual performance profiles that predict the value of the planner's current solution given the time spent planning, the state of the planning algorithm's internal parameters, and the difficulty of the planning problem being solved. This model removes the need to assume that the current solution quality is always known, broadening the class of metareasoning problems that can be addressed. We then employ deep reinforcement learning to learn a policy that decides, at each timestep, whether to continue planning or start executing the current plan, and how to set hyperparameters of the planner to enhance its performance. We demonstrate our algorithm's ability to perform effective metareasoning in two domains.Downloads
Published
2024-03-24
How to Cite
Budd, M., Lacerda, B., & Hawes, N. (2024). Stop! Planner Time: Metareasoning for Probabilistic Planning Using Learned Performance Profiles. Proceedings of the AAAI Conference on Artificial Intelligence, 38(18), 20053-20060. https://doi.org/10.1609/aaai.v38i18.29983
Issue
Section
AAAI Technical Track on Planning, Routing, and Scheduling