Google Research Football: A Novel Reinforcement Learning Environment

Authors

  • Karol Kurach Google
  • Anton Raichuk Google
  • Piotr Stańczyk Google
  • Michał Zając Google
  • Olivier Bachem Google
  • Lasse Espeholt Google
  • Carlos Riquelme Google
  • Damien Vincent Google
  • Marcin Michalski Google
  • Olivier Bousquet Google
  • Sylvain Gelly Google

DOI:

https://doi.org/10.1609/aaai.v34i04.5878

Abstract

Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly tested in a safe and reproducible manner. We introduce the Google Research Football Environment, a new reinforcement learning environment where agents are trained to play football in an advanced, physics-based 3D simulator. The resulting environment is challenging, easy to use and customize, and it is available under a permissive open-source license. In addition, it provides support for multiplayer and multi-agent experiments. We propose three full-game scenarios of varying difficulty with the Football Benchmarks and report baseline results for three commonly used reinforcement algorithms (IMPALA, PPO, and Ape-X DQN). We also provide a diverse set of simpler scenarios with the Football Academy and showcase several promising research directions.

Downloads

Published

2020-04-03

How to Cite

Kurach, K., Raichuk, A., Stańczyk, P., Zając, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bousquet, O., & Gelly, S. (2020). Google Research Football: A Novel Reinforcement Learning Environment. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 4501-4510. https://doi.org/10.1609/aaai.v34i04.5878

Issue

Section

AAAI Technical Track: Machine Learning