Amata: An Annealing Mechanism for Adversarial Training Acceleration

Authors

  • Nanyang Ye Shanghai Jiao Tong University
  • Qianxiao Li National University of Singapore
  • Xiao-Yun Zhou The Hamlyn Centre for Robotic Surgery, Imperial College London
  • Zhanxing Zhu Peking University

Keywords:

(Deep) Neural Network Algorithms

Abstract

Despite the empirical success in various domains, it has been revealed that deep neural networks are vulnerable to maliciously perturbed input data that much degrade their performance. This is known as adversarial attacks. To counter adversarial attacks, adversarial training formulated as a form of robust optimization has been demonstrated to be effective. However, conducting adversarial training brings much computational overhead compared with standard training. In order to reduce the computational cost, we propose an annealing mechanism, Amata, to reduce the overhead associated with adversarial training. The proposed Amata is provably convergent, well-motivated from the lens of optimal control theory and can be combined with existing acceleration methods to further enhance performance. It is demonstrated that on standard datasets, Amata can achieve similar or better robustness with around 1/3 to 1/2 the computational time compared with traditional methods. In addition, Amata can be incorporated into other adversarial training acceleration algorithms (e.g. YOPO, Free, Fast, and ATTA), which leads to further reduction in computational time on large-scale problems.

Downloads

Published

2021-05-18

How to Cite

Ye, N., Li, Q., Zhou, X.-Y., & Zhu, Z. (2021). Amata: An Annealing Mechanism for Adversarial Training Acceleration. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 10691-10699. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17278

Issue

Section

AAAI Technical Track on Machine Learning V