Point-Based POMDP Solving with Factored Value Function Approximation


  • Tiago Veiga Universidade de Lisboa
  • Matthijs Spaan Delft University of Technology
  • Pedro Lima Universidade de Lisboa




Partially observable Markov decision processes (POMDPs) provide a principled mathematical framework for modeling autonomous decision-making problems. A POMDP solution is often represented by a value function comprised of a set of vectors. In the case of factored models, the size of these vectors grows exponentially with the number of state factors, leading to scalability issues. We consider an approximate value function representation based on a linear combination of basis functions. In particular, we present a backup operator that can be used in any point-based POMDP solver. Furthermore, we show how under certain conditions independence between observation factors can be exploited for large computational gains. We experimentally verify our contributions and show that they have the potential to improve point-based methods in policy quality and solution size.




How to Cite

Veiga, T., Spaan, M., & Lima, P. (2014). Point-Based POMDP Solving with Factored Value Function Approximation. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1). https://doi.org/10.1609/aaai.v28i1.9070



AAAI Technical Track: Reasoning under Uncertainty