TY - JOUR
AU - Xiong, Haoyi
AU - Wang, Kafeng
AU - Bian, Jiang
AU - Zhu, Zhanxing
AU - Xu, Cheng-Zhong
AU - Guo, Zhishan
AU - Huan, Jun
PY - 2019/07/17
Y2 - 2024/05/29
TI - SpHMC: Spectral Hamiltonian Monte Carlo
JF - Proceedings of the AAAI Conference on Artificial Intelligence
JA - AAAI
VL - 33
IS - 01
SE - AAAI Technical Track: Machine Learning
DO - 10.1609/aaai.v33i01.33015516
UR - https://ojs.aaai.org/index.php/AAAI/article/view/4492
SP - 5516-5524
AB - <p>Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) methods have been widely used to sample from certain probability distributions, incorporating (kernel) density derivatives and/or given datasets. Instead of exploring new samples from kernel spaces, this piece of work proposed a novel SGHMC sampler, namely <em>Spectral Hamiltonian Monte Carlo</em> (SpHMC), that produces the high dimensional sparse representations of given datasets through sparse sensing and SGHMC. Inspired by compressed sensing, we assume all given samples are low-dimensional measurements of certain high-dimensional sparse vectors, while a continuous probability distribution exists in such high-dimensional space. Specifically, given a dictionary for sparse coding, SpHMC first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolisâ€“Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the dictionary. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of SpHMC beyond baseline methods.</p>
ER -