HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors
DOI:
https://doi.org/10.1609/aaai.v38i15.29615Keywords:
ML: Semi-Supervised Learning, ML: Active Learning, ML: Classification and Regression, ML: Multi-class/Multi-label Learning & Extreme Classification, ML: Transfer, Domain Adaptation, Multi-Task LearningAbstract
Graph Attention Networks (GATs) that compute node representation by its lower-order neighbors, are state-of-the-art architecture for representation learning with graphs. In practice, however, the high-order neighbors that turn out to be useful, remain largely unemployed in GATs. Efforts on this issue remain to be limited. This paper proposes a simple and effective high-order neighbor GAT (HONGAT) model to both effectively exploit informative high-order neighbors and address over-smoothing at the decision boundary of nodes. Two tightly coupled novel technologies, namely common neighbor similarity and new masking matrix, are introduced. Specifically, high-order neighbors are fully explored by generic high-order common-neighbor-based similarity; in order to prevent severe over-smoothing, typical averaging range no longer works well and a new masking mechanism is employed without any extra hyperparameter. Extensive empirical evaluation on real-world datasets clearly shows the necessity of the new algorithm in the ability of exploring high-order neighbors, which promisingly achieves significant gains over previous state-of-the-art graph attention methods.Downloads
Published
2024-03-24
How to Cite
Zhang, H.-K., Zhang, Y.-G., Zhou, Z., & Li, Y.-F. (2024). HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15), 16750-16758. https://doi.org/10.1609/aaai.v38i15.29615
Issue
Section
AAAI Technical Track on Machine Learning VI