The Effect of Diversity in Meta-Learning
DOI:
https://doi.org/10.1609/aaai.v37i7.26012Keywords:
ML: Meta Learning, ML: Multi-Instance/Multi-View Learning, ML: Representation Learning, ML: Transfer, Domain Adaptation, Multi-Task Learning, ML: Active Learning, ML: Optimization, ML: Deep Neural Network Algorithms, ML: Other Foundations of Machine Learning, ML: Classification and Regression, ML: Evaluation and Analysis (Machine Learning)Abstract
Recent studies show that task distribution plays a vital role in the meta-learner's performance. Conventional wisdom is that task diversity should improve the performance of meta-learning. In this work, we find evidence to the contrary; (i) our experiments draw into question the efficacy of our learned models: similar manifolds can be learned with a subset of the data (lower task diversity). This finding questions the advantage of providing more data to the model, and (ii) adding diversity to the task distribution (higher task diversity) sometimes hinders the model and does not lead to a significant improvement in performance as previously believed. To strengthen our findings, we provide both empirical and theoretical evidence.
Downloads
Published
2023-06-26
How to Cite
Kumar, R., Deleu, T., & Bengio, Y. (2023). The Effect of Diversity in Meta-Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7), 8396-8404. https://doi.org/10.1609/aaai.v37i7.26012
Issue
Section
AAAI Technical Track on Machine Learning II
