Deep Innovation Protection: Confronting the Credit Assignment Problem in Training Heterogeneous Neural Architectures
Keywords:Evolutionary Computation, Bio-inspired Learning, (Deep) Neural Network Algorithms
AbstractDeep reinforcement learning approaches have shown impressive results in a variety of different domains, however, more complex heterogeneous architectures such as world models require the different neural components to be trained separately instead of end-to-end. While a simple genetic algorithm recently showed end-to-end training is possible, it failed to solve a more complex 3D task. This paper presents a method called Deep Innovation Protection (DIP) that addresses the credit assignment problem in training complex heterogenous neural network models end-to-end for such environments. The main idea behind the approach is to employ multiobjective optimization to temporally reduce the selection pressure on specific components in multi-component network, allowing other components to adapt. We investigate the emergent representations of these evolved networks, which learn to predict properties important for the survival of the agent, without the need for a specific forward-prediction loss.
How to Cite
Risi, S., & Stanley, K. O. (2021). Deep Innovation Protection: Confronting the Credit Assignment Problem in Training Heterogeneous Neural Architectures. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 12391-12399. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17470
AAAI Technical Track on Search and Optimization