HoD-Net: High-Order Differentiable Deep Neural Networks and Applications
DOI:
https://doi.org/10.1609/aaai.v36i8.20799Keywords:
Machine Learning (ML)Abstract
We introduce a deep architecture named HoD-Net to enable high-order differentiability for deep learning. HoD-Net is based on and generalizes the complex-step finite difference (CSFD) method. While similar to classic finite difference, CSFD approaches the derivative of a function from a higher-dimension complex domain, leading to highly accurate and robust differentiation computation without numerical stability issues. This method can be coupled with backpropagation and adjoint perturbation methods for an efficient calculation of high-order derivatives. We show how this numerical scheme can be leveraged in challenging deep learning problems, such as high-order network training, deep learning-based physics simulation, and neural differential equations.Downloads
Published
2022-06-28
How to Cite
Shen, S., Shao, T., Zhou, K., Jiang, C., Luo, F., & Yang, Y. (2022). HoD-Net: High-Order Differentiable Deep Neural Networks and Applications. Proceedings of the AAAI Conference on Artificial Intelligence, 36(8), 8249-8258. https://doi.org/10.1609/aaai.v36i8.20799
Issue
Section
AAAI Technical Track on Machine Learning III