Using Hindsight to Anchor Past Knowledge in Continual Learning

Authors

  • Arslan Chaudhry University of Oxford
  • Albert Gordo Facebook AI
  • Puneet Dokania University of Oxford
  • Philip Torr University of Oxford
  • David Lopez-Paz Facebook AI

Keywords:

Transfer/Adaptation/Multi-task/Meta/Automated Learning

Abstract

In continual learning, the learner faces a stream of data whose distribution changes over time. Modern neural networks are known to suffer under this setting, as they quickly forget previously acquired knowledge. To address such catastrophic forgetting, many continual learning methods implement different types of experience replay, re-learning on past data stored in a small buffer known as episodic memory. In this work, we complement experience replay with a new objective that we call ``anchoring'', where the learner uses bilevel optimization to update its knowledge on the current task, while keeping intact the predictions on some anchor points of past tasks. These anchor points are learned using gradient-based optimization to maximize forgetting, which is approximated by fine-tuning the currently trained model on the episodic memory of past tasks. Experiments on several supervised learning benchmarks for continual learning demonstrate that our approach improves the standard experience replay in terms of both accuracy and forgetting metrics and for various sizes of episodic memory.

Downloads

Published

2021-05-18

How to Cite

Chaudhry, A., Gordo, A., Dokania, P., Torr, P., & Lopez-Paz, D. (2021). Using Hindsight to Anchor Past Knowledge in Continual Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 6993-7001. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16861

Issue

Section

AAAI Technical Track on Machine Learning I