Memory-Augmented Monte Carlo Tree Search

Authors

  • Chenjun Xiao University of Alberta
  • Jincheng Mei University of Alberta
  • Martin Müller University of Alberta

Keywords:

Monte Carlo tree search, Memory, Value function estimation

Abstract

This paper proposes and evaluates Memory-Augmented Monte Carlo Tree Search (M-MCTS), which provides a new approach to exploit generalization in online real-time search. The key idea of M-MCTS is to incorporate MCTS with a memory structure, where each entry contains information of a particular state. This memory is used to generate an approximate value estimation by combining the estimations of similar states. We show that the memory based value approximation is better than the vanilla Monte Carlo estimation with high probability under mild conditions. We evaluate M-MCTS in the game of Go. Experimental results show that M-MCTS outperforms the original MCTS with the same number of simulations.

Downloads

Published

2018-04-25

How to Cite

Xiao, C., Mei, J., & Müller, M. (2018). Memory-Augmented Monte Carlo Tree Search. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/11531

Issue

Section

AAAI Technical Track: Heuristic Search and Optimization