Fast Incremental Policy Compilation from Plans in Hybrid Probabilistic Domains

Authors

  • Florent Teichteil-Königsbuch ONERA

DOI:

https://doi.org/10.1609/icaps.v22i1.13519

Keywords:

Hybrid Markov Decision Processes, Determinization-based Probabilistic Planning

Abstract

We present the domain-independent HRFF algorithm, which solves goal-oriented HMDPs by incrementally aggregating plans generated by the METRIC-FF planner into a policy defined over discrete and continuous state variables. HRFF takes into account non-monotonic state variables, and complex combinations of many discrete and continuous probability distributions. We introduce new data structures and algorithmic paradigms to deal with continuous state spaces: hybrid hierarchical hash tables, domain determinization based on dynamic domain sampling or on static computation of probability distributions' modes, optimization settings under METRIC-FF based on plan probability and length. We deeply analyze the behavior of HRFF on a probabilistically-interesting structured navigation problem with continuous dead-ends and non-monotonic continuous state variables. We compare with HAO* on the Rover domain and show that HRFF outperforms HAO* by many order of magnitudes in terms of computation time and memory usage. We also experiment challenging and combinatorial HMDP versions of benchmarks from numeric classical planning.

Downloads

Published

2012-05-14

How to Cite

Teichteil-Königsbuch, F. (2012). Fast Incremental Policy Compilation from Plans in Hybrid Probabilistic Domains. Proceedings of the International Conference on Automated Planning and Scheduling, 22(1), 252-260. https://doi.org/10.1609/icaps.v22i1.13519