LB-DESPOT: Efficient Online POMDP Planning Considering Lower Bound in Action Selection (Student Abstract)

Authors

  • Chenyang Wu National Key Lab for Novel Software Technology, Nanjing University
  • Rui Kong National Key Lab for Novel Software Technology, Nanjing University
  • Guoyu Yang National Key Lab for Novel Software Technology, Nanjing University
  • Xianghan Kong National Key Lab for Novel Software Technology, Nanjing University
  • Zongzhang Zhang National Key Lab for Novel Software Technology, Nanjing University
  • Yang Yu National Key Lab for Novel Software Technology, Nanjing University
  • Dong Li Noah’s Ark Lab, Huawei Company
  • Wulong Liu Noah’s Ark Lab, Huawei Company

Keywords:

Planning Under Uncertainty, POMDP, Online Planning

Abstract

Partially observable Markov decision process (POMDP) is an extension to MDP. It handles the state uncertainty by specifying the probability of getting a particular observation given the current state. DESPOT is one of the most popular scalable online planning algorithms for POMDPs, which manages to significantly reduce the size of the decision tree while deriving a near-optimal policy by considering only $K$ scenarios. Nevertheless, there is a gap in action selection criteria between planning and execution in DESPOT. During the planning stage, it keeps choosing the action with the highest upper bound, whereas when the planning ends, the action with the highest lower bound is chosen for execution. Here, we propose LB-DESPOT to alleviate this issue, which utilizes the lower bound in selecting an action branch to expand. Empirically, our method has attained better performance than DESPOT and POMCP, which is another state-of-the-art, on several challenging POMDP benchmark tasks.

Downloads

Published

2021-05-18

How to Cite

Wu, C., Kong, R., Yang, G., Kong, X., Zhang, Z., Yu, Y., Li, D., & Liu, W. (2021). LB-DESPOT: Efficient Online POMDP Planning Considering Lower Bound in Action Selection (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15927-15928. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17960

Issue

Section

AAAI Student Abstract and Poster Program