Synthesis of Search Heuristics for Temporal Planning via Reinforcement Learning


  • Andrea Micheli Fondazione Bruno Kessler
  • Alessandro Valentini Fondazione Bruno Kessler



Planning/Scheduling and Learning, Temporal Planning


Automated temporal planning is the problem of synthesizing, starting from a model of a system, a course of actions to achieve a desired goal when temporal constraints, such as deadlines, are present in the problem. Despite considerable successes in the literature, scalability is still a severe limitation for existing planners, especially when confronted with real-world, industrial scenarios. In this paper, we aim at exploiting recent advances in reinforcement learning, for the synthesis of heuristics for temporal planning. Starting from a set of problems of interest for a specific domain, we use a customized reinforcement learning algorithm to construct a value function that is able to estimate the expected reward for as many problems as possible. We use a reward schema that captures the semantics of the temporal planning problem and we show how the value function can be transformed in a planning heuristic for a semi-symbolic heuristic search exploration of the planning model. We show on two case-studies how this method can widen the reach of current temporal planners with encouraging results.




How to Cite

Micheli, A., & Valentini, A. (2021). Synthesis of Search Heuristics for Temporal Planning via Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13), 11895-11902.



AAAI Technical Track on Planning, Routing, and Scheduling