Generalized Value Iteration Networks:Life Beyond Lattices

Authors

  • Sufeng Niu Clemson University
  • Siheng Chen Uber Advanced Technologies Group
  • Hanyu Guo Clemson University
  • Colin Targonski Clemson University
  • Melissa Smith Clemson University
  • Jelena Kovačević Carnegie Mellon University

Keywords:

value iteration network, irregular graph, graph convolution

Abstract

In this paper, we introduce a generalized value iteration network (GVIN), which is an end-to-end neural network planning module. GVIN emulates the value iteration algorithm by using a novel graph convolution operator, which enables GVIN to learn and plan on irregular spatial graphs. We propose three novel differentiable kernels as graph convolution operators and show that the embedding-based kernel achieves the best performance. Furthermore, we present episodic Q-learning, an improvement upon traditional n-step Q-learning that stabilizes training for VIN and GVIN. Lastly, we evaluate GVIN on planning problems in 2D mazes, irregular graphs, and real-world street networks, showing that GVIN generalizes well for both arbitrary graphs and unseen graphs of larger scaleand outperforms a naive generalization of VIN (discretizing a spatial graph into a 2D image).

Downloads

Published

2018-04-26

How to Cite

Niu, S., Chen, S., Guo, H., Targonski, C., Smith, M., & Kovačević, J. (2018). Generalized Value Iteration Networks:Life Beyond Lattices. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/12081