Meta-Learning Framework with Applications to Zero-Shot Time-Series Forecasting

Authors

  • Boris N. Oreshkin Element AI
  • Dmitri Carpov Element AI
  • Nicolas Chapados Element AI
  • Yoshua Bengio Mila

Keywords:

Time-Series/Data Streams, Transfer/Adaptation/Multi-task/Meta/Automated Learning, Mining of Spatial, Temporal or Spatio-Temporal Da, (Deep) Neural Network Algorithms

Abstract

Can meta-learning discover generic ways of processing time series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets? This work provides positive evidence to this using a broad meta-learning framework which we show subsumes many existing meta-learning algorithms. Our theoretical analysis suggests that residual connections act as a meta-learning adaptation mechanism, generating a subset of task-specific parameters based on a given TS input, thus gradually expanding the expressive power of the architecture on-the-fly. The same mechanism is shown via linearization analysis to have the interpretation of a sequential update of the final linear layer. Our empirical results on a wide range of data emphasize the importance of the identified meta-learning mechanisms for successful zero-shot univariate forecasting, suggesting that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining, resulting in performance that is at least as good as that of state-of-practice univariate forecasting models.

Downloads

Published

2021-05-18

How to Cite

Oreshkin, B. . N., Carpov, D., Chapados, N., & Bengio, Y. (2021). Meta-Learning Framework with Applications to Zero-Shot Time-Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35(10), 9242-9250. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17115

Issue

Section

AAAI Technical Track on Machine Learning III