On the Challenges of Physical Implementations of RBMs

Authors

  • Vincent Dumoulin Université de Montréal
  • Ian Goodfellow Université de Montréal
  • Aaron Courville Université de Montréal
  • Yoshua Bengio Université de Montréal

DOI:

https://doi.org/10.1609/aaai.v28i1.8924

Keywords:

RBM, restricted Boltzmann machine, physical computing

Abstract

Restricted Boltzmann machines (RBMs) are powerful machine learning models, but learning and some kinds of inference in the model require sampling-based approximations, which, in classical digital computers, are implemented using expensive MCMC. Physical computation offers the opportunity to reduce the costof sampling by building physical systems whose natural dynamics correspond to drawing samples from the desired RBM distribution. Such a system avoids the burn-in and mixing cost of a Markov chain. However, hardware implementations of this variety usually entail limitations such as low-precision and limited range of the parameters and restrictions on the size and topology of the RBM. We conduct software simulations to determine how harmful each of these restrictions is. Our simulations are based on the D-Wave Two computer, but the issues we investigate arise in most forms of physical computation.Our findings suggest that designers of new physical computing hardware and algorithms for physical computers should focus their efforts on overcoming the limitations imposed by the topology restrictions of currently existing physical computers.

Downloads

Published

2014-06-21

How to Cite

Dumoulin, V., Goodfellow, I., Courville, A., & Bengio, Y. (2014). On the Challenges of Physical Implementations of RBMs. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1). https://doi.org/10.1609/aaai.v28i1.8924

Issue

Section

Main Track: Machine Learning Applications