Semi-transductive Learning for Generalized Zero-Shot Sketch-Based Image Retrieval

Authors

  • Ce Ge Beijing University of Posts and Telecommunications
  • Jingyu Wang Beijing University of Posts and Telecommunications
  • Qi Qi Beijing University of Posts and Telecommunications
  • Haifeng Sun Beijing University of Posts and Telecommunications
  • Tong Xu Beijing University of Posts and Telecommunications
  • Jianxin Liao Beijing University of Posts and Telecommunications

DOI:

https://doi.org/10.1609/aaai.v37i6.25931

Keywords:

ML: Semi-Supervised Learning, CV: Applications, CV: Image and Video Retrieval, CV: Multi-modal Vision, CV: Representation Learning for Vision, HAI: Human-Computer Interaction, ML: Active Learning, ML: Classification and Regression, ML: Learning Preferences or Rankings, ML: Multimodal Learning, ML: Unsupervised & Self-Supervised Learning

Abstract

Sketch-based image retrieval (SBIR) is an attractive research area where freehand sketches are used as queries to retrieve relevant images. Existing solutions have advanced the task to the challenging zero-shot setting (ZS-SBIR), where the trained models are tested on new classes without seen data. However, they are prone to overfitting under a realistic scenario when the test data includes both seen and unseen classes. In this paper, we study generalized ZS-SBIR (GZS-SBIR) and propose a novel semi-transductive learning paradigm. Transductive learning is performed on the image modality to explore the potential data distribution within unseen classes, and zero-shot learning is performed on the sketch modality sharing the learned knowledge through a semi-heterogeneous architecture. A hybrid metric learning strategy is proposed to establish semantics-aware ranking property and calibrate the joint embedding space. Extensive experiments are conducted on two large-scale benchmarks and four evaluation metrics. The results show that our method is superior over the state-of-the-art competitors in the challenging GZS-SBIR task.

Downloads

Published

2023-06-26

How to Cite

Ge, C., Wang, J., Qi, Q., Sun, H., Xu, T., & Liao, J. (2023). Semi-transductive Learning for Generalized Zero-Shot Sketch-Based Image Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6), 7678-7686. https://doi.org/10.1609/aaai.v37i6.25931

Issue

Section

AAAI Technical Track on Machine Learning I