Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent
Keywords:Planning and scheduling with continuous state and action spaces, Partially observable and unobservable domains, Planning and scheduling with mixed continuous and discrete states/actions/decisions
AbstractRather than augmenting rewards with penalties for undesired behavior, Constrained Partially Observable Markov Decision Processes (CPOMDPs) plan safely by imposing inviolable hard constraint value budgets. Previous work performing online planning for CPOMDPs has only been applied to discrete action and observation spaces. In this work, we propose algorithms for online CPOMDP planning for continuous state, action, and observation spaces by combining dual ascent with progressive widening. We empirically compare the effectiveness of our proposed algorithms on continuous CPOMDPs that model both toy and real-world safety-critical problems. Additionally, we compare against the use of online solvers for continuous unconstrained POMDPs that scalarize cost constraints into rewards and highlight the limitations of the default exploration scheme.
How to Cite
Jamgochian, A., Corso, A., & Kochenderfer, M. J. (2023). Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent. Proceedings of the International Conference on Automated Planning and Scheduling, 33(1), 198-202. https://doi.org/10.1609/icaps.v33i1.27195