Decentralised Learning in Systems With Many, Many Strategic Agents

Authors

  • David Mguni PROWLER.io
  • Joel Jennings PROWLER.io
  • Enrique Munoz de Cote PROWLER.io; INAOE, Mexico

Keywords:

Multi-agent Reinforcement Learning, Stochastic Games, Large Games, Mean Field Games, Reinforcement Learning

Abstract

Although multi-agent reinforcement learning can tackle systems of strategically interacting entities, it currently fails in scalability and lacks rigorous convergence guarantees. Crucially, learning in multi-agent systems can become intractable due to the explosion in the size of the state-action space as the number of agents increases. In this paper, we propose a method for computing closed-loop optimal policies in multi-agent systems that scales independently of the number of agents. This allows us to show, for the first time, successful convergence to optimal behaviour in systems with an unbounded number of interacting adaptive learners. Studying the asymptotic regime of N-player stochastic games, we devise a learning protocol that is guaranteed to converge to equilibrium policies even when the number of agents is extremely large. Our method is model-free and completely decentralised so that each agent need only observe its local state information and its realised rewards. We validate these theoretical results by showing convergence to Nash-equilibrium policies in applications from economics and control theory with thousands of strategically interacting agents.

Downloads

Published

2018-04-26

How to Cite

Mguni, D., Jennings, J., & Munoz de Cote, E. (2018). Decentralised Learning in Systems With Many, Many Strategic Agents. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/11586

Issue

Section

AAAI Technical Track: Multiagent Systems