Overcoming Catastrophic Forgetting by Neuron-Level Plasticity Control

Authors

  • Inyoung Paik Deepbio
  • Sangjun Oh Deepbio
  • Taeyeong Kwak Deepbio
  • Injung Kim Handong Global University

DOI:

https://doi.org/10.1609/aaai.v34i04.5981

Abstract

To address the issue of catastrophic forgetting in neural networks, we propose a novel, simple, and effective solution called neuron-level plasticity control (NPC). While learning a new task, the proposed method preserves the existing knowledge from the previous tasks by controlling the plasticity of the network at the neuron level. NPC estimates the importance value of each neuron and consolidates important neurons by applying lower learning rates, rather than restricting individual connection weights to stay close to the values optimized for the previous tasks. The experimental results on the several datasets show that neuron-level consolidation is substantially more effective compared to connection-level consolidation approaches.

Downloads

Published

2020-04-03

How to Cite

Paik, I., Oh, S., Kwak, T., & Kim, I. (2020). Overcoming Catastrophic Forgetting by Neuron-Level Plasticity Control. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 5339-5346. https://doi.org/10.1609/aaai.v34i04.5981

Issue

Section

AAAI Technical Track: Machine Learning