Deep Innovation Protection: Confronting the Credit Assignment Problem in Training Heterogeneous Neural Architectures

Authors

  • Sebastian Risi IT University of Copenhagen
  • Kenneth O. Stanley Uber AI

Keywords:

Evolutionary Computation, Bio-inspired Learning, (Deep) Neural Network Algorithms

Abstract

Deep reinforcement learning approaches have shown impressive results in a variety of different domains, however, more complex heterogeneous architectures such as world models require the different neural components to be trained separately instead of end-to-end. While a simple genetic algorithm recently showed end-to-end training is possible, it failed to solve a more complex 3D task. This paper presents a method called Deep Innovation Protection (DIP) that addresses the credit assignment problem in training complex heterogenous neural network models end-to-end for such environments. The main idea behind the approach is to employ multiobjective optimization to temporally reduce the selection pressure on specific components in multi-component network, allowing other components to adapt. We investigate the emergent representations of these evolved networks, which learn to predict properties important for the survival of the agent, without the need for a specific forward-prediction loss.

Downloads

Published

2021-05-18

How to Cite

Risi, S., & Stanley, K. O. (2021). Deep Innovation Protection: Confronting the Credit Assignment Problem in Training Heterogeneous Neural Architectures. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 12391-12399. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17470

Issue

Section

AAAI Technical Track on Search and Optimization