TY - JOUR AU - Liu, Anqi AU - Reyzin, Lev AU - Ziebart, Brian PY - 2015/02/21 Y2 - 2024/03/28 TI - Shift-Pessimistic Active Learning Using Robust Bias-Aware Prediction JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 29 IS - 1 SE - Main Track: Novel Machine Learning Algorithms DO - 10.1609/aaai.v29i1.9609 UR - https://ojs.aaai.org/index.php/AAAI/article/view/9609 SP - AB - <p> Existing approaches to active learning are generally optimistic about their certainty with respect to data shift between labeled and unlabeled data. They assume that unknown datapoint labels follow the inductive biases of the active learner. As a result, the most useful datapoint labels—ones that refute current inductive biases—are rarely solicited. We propose a shift-pessimistic approach to active learning that assumes the worst-case about the unknown conditional label distribution. This closely aligns model uncertainty with generalization error, enabling more useful label solicitation. We investigate the theoretical benefits of this approach and demonstrate its empirical advantages on probabilistic binary classification tasks. </p> ER -