Decentralized High-Dimensional Bayesian Optimization With Factor Graphs

Authors

  • Trong Nghia Hoang Massachusetts Institute of Technology
  • Quang Minh Hoang National University of Singapore
  • Ruofei Ouyang National University of Singapore
  • Kian Hsiang Low National University of Singapore

Abstract

This paper presents a novel decentralized high-dimensional Bayesian optimization (DEC-HBO) algorithm that, in contrast to existing HBO algorithms, can exploit the interdependent effects of various input components on the output of the unknown objective function f for boosting the BO performance and still preserve scalability in the number of input dimensions without requiring prior knowledge or the existence of a low (effective) dimension of the input space. To realize this, we propose a sparse yet rich factor graph representation of f to be exploited for designing an acquisition function that can be similarly represented by a sparse factor graph and hence be efficiently optimized in a decentralized manner using distributed message passing. Despite richly characterizing the interdependent effects of the input components on the output of f with a factor graph, DEC-HBO can still guarantee no-regret performance asymptotically. Empirical evaluation on synthetic and real-world experiments (e.g., sparse Gaussian process model with 1811 hyperparameters) shows that DEC-HBO outperforms the state-of-the-art HBO algorithms.

Downloads

Published

2018-04-29

How to Cite

Hoang, T. N., Hoang, Q. M., Ouyang, R., & Low, K. H. (2018). Decentralized High-Dimensional Bayesian Optimization With Factor Graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/11788