Kill Two Birds With One Stone: Weakly-Supervised Neural Network for Image Annotation and Tag Refinement

Authors

  • Junjie Zhang University of Technology Sydney
  • Qi Wu University of Adelaide
  • Jian Zhang University of Technology Sydney
  • Chunhua Shen University of Adelaide
  • Jianfeng Lu Nanjing University of Science and Technology

Keywords:

Weakly-Supervised Neural Network, Social Image Annotation, Tag Refinement

Abstract

The number of social images has exploded by the wide adoption of social networks, and people like to share their comments about them. These comments can be a description of the image, or some objects, attributes, scenes in it, which are normally used as the user-provided tags. However, it is well-known that user-provided tags are incomplete and imprecise to some extent. Directly using them can damage the performance of related applications, such as the image annotation and retrieval. In this paper, we propose to learn an image annotation model and refine the user-provided tags simultaneously in a weakly-supervised manner. The deep neural network is utilized as the image feature learning and backbone annotation model, while visual consistency, semantic dependency, and user-error sparsity are introduced as the constraints at the batch level to alleviate the tag noise. Therefore, our model is highly flexible and stable to handle large-scale image sets. Experimental results on two benchmark datasets indicate that our proposed model achieves the best performance compared to the state-of-the-art methods.

Downloads

Published

2018-04-27

How to Cite

Zhang, J., Wu, Q., Zhang, J., Shen, C., & Lu, J. (2018). Kill Two Birds With One Stone: Weakly-Supervised Neural Network for Image Annotation and Tag Refinement. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/12261