Kernel-convoluted Deep Neural Networks with Data Augmentation

Authors

  • Minjin Kim Seoul National University
  • Young-geun Kim Seoul National University
  • Dongha Kim Seoul National University
  • Yongdai Kim Seoul National University
  • Myunghee Cho Paik Seoul National University

Keywords:

(Deep) Neural Network Algorithms, Classification and Regression, Kernel Methods, Learning Theory

Abstract

The Mixup method, which uses linearly interpolated data, has emerged as an effective data augmentation tool to improve generalization performance and the robustness to adversarial examples. The motivation is to curtail undesirable oscillations by its implicit model constraint to behave linearly at in-between observed data points and promote smoothness. In this work, we formally investigate this premise, propose a way to impose smoothness constraints explicitly, and extend it to incorporate implicit model constraints. First, we derive a new function class composed of kernel-convoluted models (KCM) where the smoothness constraint is directly imposed by locally averaging the original functions with a kernel function. Second, we propose to incorporate the Mixup method into KCM to expand the domains of smoothness. In both cases of the KCM and the KCM adapted with the Mixup, we provide risk analysis, respectively, under mild conditions on kernel functions. As a result, we show that the upper bound of the excess risk over a new function class is not slower than that of the excess risk over the original function class. Using CIFAR-10 and CIFAR-100 datasets, our experiments demonstrate that the KCM with the Mixup outperforms the Mixup method in terms of generalization and robustness to adversarial examples.

Downloads

Published

2021-05-18

How to Cite

Kim, M., Kim, Y.- geun, Kim, D., Kim, Y., & Paik, M. C. (2021). Kernel-convoluted Deep Neural Networks with Data Augmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 8155-8162. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16993

Issue

Section

AAAI Technical Track on Machine Learning II