NASGEM: Neural Architecture Search via Graph Embedding Method

Authors

  • Hsin-Pai Cheng Duke University
  • Tunhou Zhang Duke University
  • Yixing Zhang Duke University
  • Shiyu Li Duke University
  • Feng Liang Tsinghua University
  • Feng Yan University of Nevada, Reno
  • Meng Li Facebook Inc.
  • Vikas Chandra Facebook Inc.
  • Hai Li Duke University
  • Yiran Chen Duke University

Keywords:

(Deep) Neural Network Algorithms

Abstract

Neural Architecture Search (NAS) automates and prospers the design of neural networks. Estimator-based NAS has been proposed recently to model the relationship between architectures and their performance to enable scalable and flexible search. However, existing estimator-based methods encode the architecture into a latent space without considering graph similarity. Ignoring graph similarity in node-based search space may induce a large inconsistency between similar graphs and their distance in the continuous encoding space, leading to inaccurate encoding representation and/or reduced representation capacity that can yield sub-optimal search results. To preserve graph correlation information in encoding, we propose NASGEM which stands for Neural Architecture Search via Graph Embedding Method. NASGEM is driven by a novel graph embedding method equipped with similarity measures to capture the graph topology information. By precisely estimating the graph distance and using an auxiliary Weisfeiler-Lehman kernel to guide the encoding, NASGEM can utilize additional structural information to get more accurate graph representation to improve the search efficiency. GEMNet, a set of networks discovered by NASGEM, consistently outperforms networks crafted by existing search methods in classification tasks, i.e., with 0.4%-3.6% higher accuracy while having 11%- 21% fewer Multiply-Accumulates. We further transfer GEMNet for COCO object detection. In both one-stage and twostage detectors, our GEMNet surpasses its manually-crafted and automatically-searched counterparts.

Downloads

Published

2021-05-18

How to Cite

Cheng, H.-P., Zhang, T., Zhang, Y., Li, S., Liang, F., Yan, F., Li, M., Chandra, V., Li, H., & Chen, Y. (2021). NASGEM: Neural Architecture Search via Graph Embedding Method. Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 7090-7098. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16872

Issue

Section

AAAI Technical Track on Machine Learning I