Reborn Filters: Pruning Convolutional Neural Networks with Limited Data

Authors

  • Yehui Tang Peking University
  • Shan You SenseTime Research
  • Chang Xu The University of Sydney
  • Jin Han Peking University
  • Chen Qian SenseTime Research
  • Boxin Shi Peking University
  • Chao Xu Peking University
  • Changshui Zhang Tsinghua University

DOI:

https://doi.org/10.1609/aaai.v34i04.6058

Abstract

Channel pruning is effective in compressing the pretrained CNNs for their deployment on low-end edge devices. Most existing methods independently prune some of the original channels and need the complete original dataset to fix the performance drop after pruning. However, due to commercial protection or data privacy, users may only have access to a tiny portion of training examples, which could be insufficient for the performance recovery. In this paper, for pruning with limited data, we propose to use all original filters to directly develop new compact filters, named reborn filters, so that all useful structure priors in the original filters can be well preserved into the pruned networks, alleviating the performance drop accordingly. During training, reborn filters can be easily implemented via 1×1 convolutional layers and then be fused in the inference stage for acceleration. Based on reborn filters, the proposed channel pruning algorithm shows its effectiveness and superiority on extensive experiments.

Downloads

Published

2020-04-03

How to Cite

Tang, Y., You, S., Xu, C., Han, J., Qian, C., Shi, B., Xu, C., & Zhang, C. (2020). Reborn Filters: Pruning Convolutional Neural Networks with Limited Data. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 5972-5980. https://doi.org/10.1609/aaai.v34i04.6058

Issue

Section

AAAI Technical Track: Machine Learning