Memory-Augmented Monte Carlo Tree Search


  • Chenjun Xiao University of Alberta
  • Jincheng Mei University of Alberta
  • Martin Müller University of Alberta


Monte Carlo tree search, Memory, Value function estimation


This paper proposes and evaluates Memory-Augmented Monte Carlo Tree Search (M-MCTS), which provides a new approach to exploit generalization in online real-time search. The key idea of M-MCTS is to incorporate MCTS with a memory structure, where each entry contains information of a particular state. This memory is used to generate an approximate value estimation by combining the estimations of similar states. We show that the memory based value approximation is better than the vanilla Monte Carlo estimation with high probability under mild conditions. We evaluate M-MCTS in the game of Go. Experimental results show that M-MCTS outperforms the original MCTS with the same number of simulations.




How to Cite

Xiao, C., Mei, J., & Müller, M. (2018). Memory-Augmented Monte Carlo Tree Search. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from



AAAI Technical Track: Heuristic Search and Optimization