SpHMC: Spectral Hamiltonian Monte Carlo

Authors

  • Haoyi Xiong Baidu Inc. & National Engineering Laboratory for Deep Learning Technology and Applications
  • Kafeng Wang Chinese Academy of Sciences & University of Chinese Academy of Sciences
  • Jiang Bian University of Central Florida
  • Zhanxing Zhu Peking University & Beijing Institute of Big Data Research
  • Cheng-Zhong Xu Chinese Academy of Sciences
  • Zhishan Guo University of Central Florida
  • Jun Huan Baidu Inc. & National Engineering Laboratory for Deep Learning Technology and Applications

DOI:

https://doi.org/10.1609/aaai.v33i01.33015516

Abstract

Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) methods have been widely used to sample from certain probability distributions, incorporating (kernel) density derivatives and/or given datasets. Instead of exploring new samples from kernel spaces, this piece of work proposed a novel SGHMC sampler, namely Spectral Hamiltonian Monte Carlo (SpHMC), that produces the high dimensional sparse representations of given datasets through sparse sensing and SGHMC. Inspired by compressed sensing, we assume all given samples are low-dimensional measurements of certain high-dimensional sparse vectors, while a continuous probability distribution exists in such high-dimensional space. Specifically, given a dictionary for sparse coding, SpHMC first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolis–Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the dictionary. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of SpHMC beyond baseline methods.

Downloads

Published

2019-07-17

How to Cite

Xiong, H., Wang, K., Bian, J., Zhu, Z., Xu, C.-Z., Guo, Z., & Huan, J. (2019). SpHMC: Spectral Hamiltonian Monte Carlo. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 5516-5524. https://doi.org/10.1609/aaai.v33i01.33015516

Issue

Section

AAAI Technical Track: Machine Learning