TY - JOUR AU - Yan, Kun AU - Zhang, Chenbin AU - Hou, Jun AU - Wang, Ping AU - Bouraoui, Zied AU - Jameel, Shoaib AU - Schockaert, Steven PY - 2022/06/28 Y2 - 2024/03/28 TI - Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 36 IS - 3 SE - AAAI Technical Track on Computer Vision III DO - 10.1609/aaai.v36i3.20205 UR - https://ojs.aaai.org/index.php/AAAI/article/view/20205 SP - 2991-2999 AB - Multi-label few-shot image classification (ML-FSIC) is the task of assigning descriptive labels to previously unseen images, based on a small number of training examples. A key feature of the multi-label setting is that images often have multiple labels, which typically refer to different regions of the image. When estimating prototypes, in a metric-based setting, it is thus important to determine which regions are relevant for which labels, but the limited amount of training data makes this highly challenging. As a solution, in this paper we propose to use word embeddings as a form of prior knowledge about the meaning of the labels. In particular, visual prototypes are obtained by aggregating the local feature maps of the support images, using an attention mechanism that relies on the label embeddings. As an important advantage, our model can infer prototypes for unseen labels without the need for fine-tuning any model parameters, which demonstrates its strong generalization abilities. Experiments on COCO and PASCAL VOC furthermore show that our model substantially improves the current state-of-the-art. ER -