A Nested Bi-level Optimization Framework for Robust Few Shot Learning
Keywords:Machine Learning (ML), Computer Vision (CV), Search And Optimization (SO), Domain(s) Of Application (APP)
AbstractModel-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal.Hence, it fails to address the domain shift between base and novel classes in few-shot learning. In this work, we propose a novel robust meta-learning algorithm, NESTEDMAML, which learns to assign weights to training tasks or instances. We con-sider weights as hyper-parameters and iteratively optimize them using a small set of validation tasks set in a nested bi-level optimization approach (in contrast to the standard bi-level optimization in MAML). We then applyNESTED-MAMLin the meta-training stage, which involves (1) several tasks sampled from a distribution different from the meta-test task distribution, or (2) some data samples with noisy labels.Extensive experiments on synthetic and real-world datasets demonstrate that NESTEDMAML efficiently mitigates the effects of ”unwanted” tasks or instances, leading to significant improvement over the state-of-the-art robust meta-learning methods.
How to Cite
Killamsetty, K., Li, C., Zhao, C., Chen, F., & Iyer, R. (2022). A Nested Bi-level Optimization Framework for Robust Few Shot Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7), 7176-7184. https://doi.org/10.1609/aaai.v36i7.20678
AAAI Technical Track on Machine Learning II