TY - JOUR AU - Nguyen, Quoc Phong AU - Low, Bryan Kian Hsiang AU - Jaillet, Patrick PY - 2021/05/18 Y2 - 2024/03/28 TI - An Information-Theoretic Framework for Unifying Active Learning Problems JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 35 IS - 10 SE - AAAI Technical Track on Machine Learning III DO - 10.1609/aaai.v35i10.17102 UR - https://ojs.aaai.org/index.php/AAAI/article/view/17102 SP - 9126-9134 AB - This paper presents an information-theoretic framework for unifying active learning problems: level set estimation (LSE), Bayesian optimization (BO), and their generalized variant. We first introduce a novel active learning criterion that subsumes an existing LSE algorithm and achieves state-of-the-art performance in LSE problems with a continuous input domain. Then, by exploiting the relationship between LSE and BO, we design a competitive information-theoretic acquisition function for BO that has interesting connections to upper confidence bound and max-value entropy search (MES). The latter connection reveals a drawback of MES which has important implications on not only MES but also on other MES-based acquisition functions. Finally, our unifying information-theoretic framework can be applied to solve a generalized problem of LSE and BO involving multiple level sets in a data-efficient manner. We empirically evaluate the performance of our proposed algorithms using synthetic benchmark functions, a real-world dataset, and in hyperparameter tuning of machine learning models. ER -