Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling

Authors

  • Shinichi Shirakawa Yokohama National University
  • Yasushi Iwata Yokohama National University
  • Youhei Akimoto Shinshu University,¬†Institute of Engineering

Keywords:

Neural Network, Deep Learning, Information Geometric Optimization

Abstract

Deep neural networks (DNNs) are powerful machine learning models and have succeeded in various artificial intelligence tasks. Although various architectures and modules for the DNNs have been proposed, selecting and designing the appropriate network structure for a target problem is a challenging task. In this paper, we propose a method to simultaneously optimize the network structure and weight parameters during neural network training. We consider a probability distribution that generates network structures, and optimize the parameters of the distribution instead of directly optimizing the network structure. The proposed method can apply to the various network structure optimization problems under the same framework. We apply the proposed method to several structure optimization problems such as selection of layers, selection of unit types, and selection of connections using the MNIST, CIFAR-10, and CIFAR-100 datasets. The experimental results show that the proposed method can find the appropriate and competitive network structures.

Downloads

Published

2018-04-29

How to Cite

Shirakawa, S., Iwata, Y., & Akimoto, Y. (2018). Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/11683