Sparse Boltzmann Machines with Structure Learning as Applied to Text Analysis

Authors

  • Zhourong Chen The Hong Kong University of Science and Technology
  • Nevin Zhang The Hong Kong University of Science and Technology
  • Dit-Yan Yeung The Hong Kong University of Science and Technology
  • Peixian Chen The Hong Kong University of Science and Technology

DOI:

https://doi.org/10.1609/aaai.v31i1.10773

Keywords:

RBM, Structure Learning, Sparse Boltzmann Machines, Text analysis, Neural Network

Abstract

We are interested in exploring the possibility and benefits of structure learning for deep models. As the first step, this paper investigates the matter for Restricted Boltzmann Machines (RBMs). We conduct the study with Replicated Softmax, a variant of RBMs for unsupervised text analysis. We present a method for learning what we call Sparse Boltzmann Machines, where each hidden unit is connected to a subset of the visible units instead of all of them.  Empirical results show that the method yields models with significantly improved model fit and interpretability as compared with RBMs where each hidden unit is connected to all visible units.

AAAI-2017 Proceedings Cover

Downloads

Published

2017-02-13

How to Cite

Chen, Z., Zhang, N., Yeung, D.-Y., & Chen, P. (2017). Sparse Boltzmann Machines with Structure Learning as Applied to Text Analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10773

Issue

Vol. 31 No. 1 (2017): Thirty-First AAAI Conference on Artificial Intelligence

Section

Machine Learning Methods