Residual Neural Processes

Authors

  • Byung-Jun Lee KAIST
  • Seunghoon Hong KAIST
  • Kee-Eung Kim KAIST

DOI:

https://doi.org/10.1609/aaai.v34i04.5883

Abstract

A Neural Process (NP) is a map from a set of observed input-output pairs to a predictive distribution over functions, which is designed to mimic other stochastic processes' inference mechanisms. NPs are shown to work effectively in tasks that require complex distributions, where traditional stochastic processes struggle, e.g. image completion tasks. This paper concerns the practical capacity of set function approximators despite their universality. By delving deeper into the relationship between an NP and a Bayesian last layer (BLL), it is possible to see that NPs may struggle in simple examples, which other stochastic processes can easily solve. In this paper, we propose a simple yet effective remedy; the Residual Neural Process (RNP) that leverages traditional BLL for faster training and better prediction. We demonstrate that the RNP shows faster convergence and better performance, both qualitatively and quantitatively.

Downloads

Published

2020-04-03

How to Cite

Lee, B.-J., Hong, S., & Kim, K.-E. (2020). Residual Neural Processes. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 4545-4552. https://doi.org/10.1609/aaai.v34i04.5883

Issue

Section

AAAI Technical Track: Machine Learning