GraphMix: Improved Training of GNNs for Semi-Supervised Learning

Authors

  • Vikas Verma Aalto University Mila - Québec Artificial Intelligence Institute, Montréal, Canada
  • Meng Qu Mila - Québec Artificial Intelligence Institute, Montréal, Canada,
  • Kenji Kawaguchi Massachusetts Institute ofTechnology (MIT), USA
  • Alex Lamb Universite de Montreal
  • Yoshua Bengio Mila - Québec Artificial Intelligence Institute, Montréal, Canada,
  • Juho Kannala Aalto University
  • Jian Tang Mila - Québec Artificial Intelligence Institute, Montréal, Canada,

Keywords:

(Deep) Neural Network Algorithms, Representation Learning, Graph-based Machine Learning, Semi-Supervised Learning

Abstract

We present GraphMix, a regularization method for Graph Neural Network based semi-supervised object classification, whereby we propose to train a fully-connected network jointly with the graph neural network via parameter sharing and interpolation-based regularization. Further, we provide a theoretical analysis of how GraphMix improves the generalization bounds of the underlying graph neural network, without making any assumptions about the "aggregation" layer or the depth of the graph neural networks. We experimentally validate this analysis by applying GraphMix to various architectures such as Graph Convolutional Networks, Graph Attention Networks and Graph-U-Net. Despite its simplicity, we demonstrate that GraphMix can consistently improve or closely match state-of-the-art performance using even simpler architectures such as Graph Convolutional Networks, across three established graph benchmarks: Cora, Citeseer and Pubmed citation network datasets, as well as three newly proposed datasets: Cora-Full, Co-author-CS and Co-author-Physics.

Downloads

Published

2021-05-18

How to Cite

Verma, V., Qu, M., Kawaguchi, K., Lamb, A., Bengio, Y., Kannala, J., & Tang, J. (2021). GraphMix: Improved Training of GNNs for Semi-Supervised Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(11), 10024-10032. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17203

Issue

Section

AAAI Technical Track on Machine Learning IV