Learning Step Size Controllers for Robust Neural Network Training

Authors

  • Christian Daniel TU Darmstadt
  • Jonathan Taylor Microsoft Research
  • Sebastian Nowozin Microsoft Research

Keywords:

Neural Networks, Deep Learning, Reinforcement Learning

Abstract

This paper investigates algorithms to automatically adapt the learning rate of neural networks (NNs). Starting with stochastic gradient descent, a large variety of learning methods has been proposed for the NN setting. However, these methods are usually sensitive to the initial learning rate which has to be chosen by the experimenter. We investigate several features and show how an adaptive controller can adjust the learning rate without prior knowledge of the learning problem at hand.

Downloads

Published

2016-02-21

How to Cite

Daniel, C., Taylor, J., & Nowozin, S. (2016). Learning Step Size Controllers for Robust Neural Network Training. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/10187

Issue

Section

Technical Papers: Machine Learning Methods