Characterizing Deep Gaussian Processes via Nonlinear Recurrence Systems

Authors

  • Anh Tong Ulsan National Institute of Science and Technology
  • Jaesik Choi Korea Advanced Institute of Science and Technology INEEJI

Keywords:

Bayesian Learning

Abstract

Recent advances in Deep Gaussian Processes (DGPs) show the potential to have more expressive representation than that of traditional Gaussian Processes (GPs). However, there exists a pathology of deep Gaussian processes that their learning capacities reduce significantly when the number of layers increases. In this paper, we present a new analysis in DGPs by studying its corresponding nonlinear dynamic systems to explain the issue. Existing work reports the pathology for the squared exponential kernel function. We extend our investigation to four types of common stationary kernel functions. The recurrence relations between layers are analytically derived, providing a tighter bound and the rate of convergence of the dynamic systems. We demonstrate our finding with a number of experimental results.

Downloads

Published

2021-05-18

How to Cite

Tong, A., & Choi, J. (2021). Characterizing Deep Gaussian Processes via Nonlinear Recurrence Systems. Proceedings of the AAAI Conference on Artificial Intelligence, 35(11), 9915-9922. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17191

Issue

Section

AAAI Technical Track on Machine Learning IV