Slimmable Generative Adversarial Networks

Authors

  • Liang Hou CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Zehuan Yuan ByteDance AI Lab
  • Lei Huang SKLSDE, Institute of Artificial Intelligence, Beihang University
  • Huawei Shen CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Xueqi Cheng CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Changhu Wang ByteDance AI Lab

Keywords:

Neural Generative Models & Autoencoders, Learning on the Edge & Model Compression, Computational Photography, Image & Video Synthesis

Abstract

Generative adversarial networks (GANs) have achieved remarkable progress in recent years, but the continuously growing scale of models make them challenging to deploy widely in practical applications. In particular, for real-time generation tasks, different devices require generators of different sizes due to varying computing power. In this paper, we introduce slimmable GANs (SlimGANs), which can flexibly switch the width of the generator to accommodate various quality-efficiency trade-offs at runtime. Specifically, we leverage multiple discriminators that share partial parameters to train the slimmable generator. To facilitate the consistency between generators of different widths, we present a stepwise inplace distillation technique that encourages narrow generators to learn from wide ones. As for class-conditional generation, we propose a sliceable conditional batch normalization that incorporates the label information into different widths. Our methods are validated, both quantitatively and qualitatively, by extensive experiments and a detailed ablation study.

Downloads

Published

2021-05-18

How to Cite

Hou, L., Yuan, Z., Huang, L., Shen, H., Cheng, X., & Wang, C. (2021). Slimmable Generative Adversarial Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 7746-7753. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16946

Issue

Section

AAAI Technical Track on Machine Learning II