Bi-CMR: Bidirectional Reinforcement Guided Hashing for Effective Cross-Modal Retrieval

Authors

  • Tieying Li Northeastern University, China
  • Xiaochun Yang Northeastern University, China
  • Bin Wang Northeastern University, China
  • Chong Xi Northeastern University, China
  • Hanzhong Zheng Northeastern University, China
  • Xiangmin Zhou RMIT University, Australia

DOI:

https://doi.org/10.1609/aaai.v36i9.21268

Keywords:

Search And Optimization (SO)

Abstract

Cross-modal hashing has attracted considerable attention for large-scale multimodal data. Recent supervised cross-modal hashing methods using multi-label networks utilize the semantics of multi-labels to enhance retrieval accuracy, where label hash codes are learned independently. However, all these methods assume that label annotations reliably reflect the relevance between their corresponding instances, which is not true in real applications. In this paper, we propose a novel framework called Bidirectional Reinforcement Guided Hashing for Effective Cross-Modal Retrieval (Bi-CMR), which exploits a bidirectional learning to relieve the negative impact of this assumption. Specifically, in the forward learning procedure, we highlight the representative labels and learn the reinforced multi-label hash codes by intra-modal semantic information, and further adjust similarity matrix. In the backward learning procedure, the reinforced multi-label hash codes and adjusted similarity matrix are used to guide the matching of instances. We construct two datasets with explicit relevance labels that reflect the semantic relevance of instance pairs based on two benchmark datasets. The Bi-CMR is evaluated by conducting extensive experiments over these two datasets. Experimental results prove the superiority of Bi-CMR over four state-of-the-art methods in terms of effectiveness.

Downloads

Published

2022-06-28

How to Cite

Li, T., Yang, X., Wang, B., Xi, C., Zheng, H., & Zhou, X. (2022). Bi-CMR: Bidirectional Reinforcement Guided Hashing for Effective Cross-Modal Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 36(9), 10275-10282. https://doi.org/10.1609/aaai.v36i9.21268

Issue

Section

AAAI Technical Track on Search and Optimization