Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs

Authors

  • Philipp Robbel Massachusetts Institute of Technology
  • Frans Oliehoek University of Amsterdam and University of Liverpool
  • Mykel Kochenderfer Stanford University

DOI:

https://doi.org/10.1609/aaai.v30i1.10133

Keywords:

Multiagent MDP, Factored MDP, Approximate Linear Programming, Variable Elimination, planning

Abstract

Many solution methods for Markov Decision Processes (MDPs) exploit structure in the problem and are based on value function factorization. Especially multiagent settings, however, are known to suffer from an exponential increase in value component sizes as interactions become denser, restricting problem sizes and types that can be handled. We present an approach to mitigate this limitation for certain types of multiagent systems, exploiting a property that can be thought of as "anonymous influence" in the factored MDP. We show how representational benefits from anonymity translate into computational efficiencies, both for variable elimination in a factor graph and for the approximate linear programming solution to factored MDPs. Our methods scale to factored MDPs that were previously unsolvable, such as the control of a stochastic disease process over densely connected graphs with 50 nodes and 25 agents.

Downloads

Published

2016-03-03

How to Cite

Robbel, P., Oliehoek, F., & Kochenderfer, M. (2016). Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). https://doi.org/10.1609/aaai.v30i1.10133

Issue

Section

Technical Papers: Multiagent Systems