A Novel Single-DBN Generative Model for Optimizing POMDP Controllers by Probabilistic Inference


  • Igor Kiselev University of Waterloo
  • Pascal Poupart University of Waterloo




POMDP planning, Finite State Controllers, probabilistic inference, control policy, Loopy Belief Propagation algorithm, Factored Frontier algorithm


As a promising alternative to using standard (often intractable) planning techniques with Bellman equations, we propose an interesting method of optimizing POMDP controllers by probabilistic inference in a novel equivalent single-DBN generative model. Our inference approach to POMDP planning allows for (1) for application of various techniques for probabilistic inference in single graphical models, and (2) for exploiting the factored structure in a controller architecture to take advantage of natural structural constrains of planning problems and represent them compactly. Our contributions can be summarized as follows: (1) we designed a novel single-DBN generative model that ensures that the task of probabilistic inference is equivalent to the original problem of optimizing POMDP controllers, and (2) we developed several inference approaches to approximate the value of the policy when exact inference methods are not tractable to solve large-size problems with complex graphical models. The proposed approaches to policy optimization by probabilistic inference are evaluated on several POMDP benchmark problems and the performance of the implemented approximation algorithms is compared.




How to Cite

Kiselev, I., & Poupart, P. (2014). A Novel Single-DBN Generative Model for Optimizing POMDP Controllers by Probabilistic Inference. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1). https://doi.org/10.1609/aaai.v28i1.9100