Structured Inference Networks for Nonlinear State Space Models

Authors

  • Rahul Krishnan New York University
  • Uri Shalit New York University
  • David Sontag New York University

DOI:

https://doi.org/10.1609/aaai.v31i1.10779

Keywords:

Unsupervised Learning, Deep Learning, Time Series, Hidden Markov Models

Abstract

Gaussian state space models have been used for decades as generative models of sequential data. They admit an intuitive probabilistic interpretation, have a simple functional form, and enjoy widespread adoption. We introduce a unified algorithm to efficiently learn a broad class of linear and non-linear state space models, including variants where the emission and transition distributions are modeled by deep neural networks. Our learning algorithm simultaneously learns a compiled inference network and the generative model, leveraging a structured variational approximation parameterized by recurrent neural networks to mimic the posterior distribution. We apply the learning algorithm to both synthetic and real-world datasets, demonstrating its scalability and versatility. We find that using the structured approximation to the posterior results in models with significantly higher held-out likelihood.

Downloads

Published

2017-02-13

How to Cite

Krishnan, R., Shalit, U., & Sontag, D. (2017). Structured Inference Networks for Nonlinear State Space Models. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10779