Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations
DOI:
https://doi.org/10.1609/aaai.v38i10.29036Keywords:
ML: Deep Learning Theory, ML: ApplicationsAbstract
Neural ordinary differential equations (NODEs), one of the most influential works of the differential equation-based deep learning, are to continuously generalize residual networks and opened a new field. They are currently utilized for various downstream tasks, e.g., image classification, time series classification, image generation, etc. Its key part is how to model the time-derivative of the hidden state, denoted dh(t)/dt. People have habitually used conventional neural network architectures, e.g., fully-connected layers followed by non-linear activations. In this paper, however, we present a neural operator-based method to define the time-derivative term. Neural operators were initially proposed to model the differential operator of partial differential equations (PDEs). Since the time-derivative of NODEs can be understood as a special type of the differential operator, our proposed method, called branched Fourier neural operator (BFNO), makes sense. In our experiments with general downstream tasks, our method significantly outperforms existing methods.Downloads
Published
2024-03-24
How to Cite
Cho, W., Cho, S., Jin, H., Jeon, J., Lee, K., Hong, S., Lee, D., Choi, J., & Park, N. (2024). Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10), 11543-11551. https://doi.org/10.1609/aaai.v38i10.29036
Issue
Section
AAAI Technical Track on Machine Learning I