Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations

Authors

  • Woojin Cho Yonsei University
  • Seunghyeon Cho Yonsei University
  • Hyundong Jin Yonsei University
  • Jinsung Jeon Yonsei University
  • Kookjin Lee Arizona State University
  • Sanghyun Hong Oregon State University
  • Dongeun Lee Texas A&M University-Commerce
  • Jonghyun Choi Yonsei University
  • Noseong Park Yonsei University

DOI:

https://doi.org/10.1609/aaai.v38i10.29036

Keywords:

ML: Deep Learning Theory, ML: Applications

Abstract

Neural ordinary differential equations (NODEs), one of the most influential works of the differential equation-based deep learning, are to continuously generalize residual networks and opened a new field. They are currently utilized for various downstream tasks, e.g., image classification, time series classification, image generation, etc. Its key part is how to model the time-derivative of the hidden state, denoted dh(t)/dt. People have habitually used conventional neural network architectures, e.g., fully-connected layers followed by non-linear activations. In this paper, however, we present a neural operator-based method to define the time-derivative term. Neural operators were initially proposed to model the differential operator of partial differential equations (PDEs). Since the time-derivative of NODEs can be understood as a special type of the differential operator, our proposed method, called branched Fourier neural operator (BFNO), makes sense. In our experiments with general downstream tasks, our method significantly outperforms existing methods.

Published

2024-03-24

How to Cite

Cho, W., Cho, S., Jin, H., Jeon, J., Lee, K., Hong, S., Lee, D., Choi, J., & Park, N. (2024). Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10), 11543-11551. https://doi.org/10.1609/aaai.v38i10.29036

Issue

Section

AAAI Technical Track on Machine Learning I