Learning from Mistakes – a Framework for Neural Architecture Search

Authors

  • Bhanu Garg University of California San Diego
  • Li Zhang Zhejiang University
  • Pradyumna Sridhara University of California San Diego
  • Ramtin Hosseini University of California San Diego
  • Eric Xing Mohamed bin Zayed University of Artificial Intelligence Carnegie Mellon University
  • Pengtao Xie University of California San Diego

DOI:

https://doi.org/10.1609/aaai.v36i9.21258

Keywords:

Search And Optimization (SO), Machine Learning (ML)

Abstract

Learning from one's mistakes is an effective human learning technique where the learners focus more on the topics where mistakes were made, so as to deepen their understanding. In this paper, we investigate if this human learning strategy can be applied in machine learning. We propose a novel machine learning method called Learning From Mistakes (LFM), wherein the learner improves its ability to learn by focusing more on the mistakes during revision. We formulate LFM as a three-stage optimization problem: 1) learner learns; 2) learner re-learns focusing on the mistakes, and; 3) learner validates its learning. We develop an efficient algorithm to solve the LFM problem. We apply the LFM framework to neural architecture search on CIFAR-10, CIFAR-100, and Imagenet. Experimental results strongly demonstrate the effectiveness of our model.

Downloads

Published

2022-06-28

How to Cite

Garg, B., Zhang, L., Sridhara, P., Hosseini, R., Xing, E., & Xie, P. (2022). Learning from Mistakes – a Framework for Neural Architecture Search. Proceedings of the AAAI Conference on Artificial Intelligence, 36(9), 10184-10192. https://doi.org/10.1609/aaai.v36i9.21258

Issue

Section

AAAI Technical Track on Search and Optimization