High-Dimensional Bayesian Optimization via Tree-Structured Additive Models

Authors

  • Eric Han School of Computing, National University of Singapore
  • Ishank Arora Indian Institute of Technology (BHU) Varanasi
  • Jonathan Scarlett School of Computing, National University of Singapore Department of Mathematics & Institute of Data Science, National University of Singapore

Keywords:

Bayesian Learning, Sequential Decision Making, Optimization, Kernel Methods

Abstract

Bayesian Optimization (BO) has shown significant success in tackling expensive low-dimensional black-box optimization problems. Many optimization problems of interest are high-dimensional, and scaling BO to such settings remains an important challenge. In this paper, we consider generalized additive models in which low-dimensional functions with overlapping subsets of variables are composed to model a high-dimensional target function. Our goal is to lower the computational resources required and facilitate faster model learning by reducing the model complexity while retaining the sample-efficiency of existing methods. Specifically, we constrain the underlying dependency graphs to tree structures in order to facilitate both the structure learning and optimization of the acquisition function. For the former, we propose a hybrid graph learning algorithm based on Gibbs sampling and mutation. In addition, we propose a novel zooming-based algorithm that permits generalized additive models to be employed more efficiently in the case of continuous domains. We demonstrate and discuss the efficacy of our approach via a range of experiments on synthetic functions and real-world datasets.

Downloads

Published

2021-05-18

How to Cite

Han, E., Arora, I., & Scarlett, J. (2021). High-Dimensional Bayesian Optimization via Tree-Structured Additive Models. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 7630-7638. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16933

Issue

Section

AAAI Technical Track on Machine Learning II