Activity Image-to-Video Retrieval by Disentangling Appearance and Motion
Keywords:Image and Video Retrieval
AbstractWith the rapid emergence of video data, image-to-video retrieval has attracted much attention. There are two types of image-to-video retrieval: instance-based and activity-based. The former task aims to retrieve videos containing the same main objects as the query image, while the latter focuses on finding the similar activity. Since dynamic information plays a significant role in the video, we pay attention to the latter task to explore the motion relation between images and videos. In this paper, we propose a Motion-assisted Activity Proposal-based Image-to-Video Retrieval (MAP-IVR) approach to disentangle the video features into motion features and appearance features and obtain appearance features from the images. Then, we perform image-to-video translation to improve the disentanglement quality. The retrieval is performed in both appearance and video feature spaces. Extensive experiments demonstrate that our MAP-IVR approach remarkably outperforms the state-of-the-art approaches on two benchmark activity-based video datasets.
How to Cite
Liu, L. ., Li, J., Niu, L., Xu, R., & Zhang, L. (2021). Activity Image-to-Video Retrieval by Disentangling Appearance and Motion. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3), 2145-2153. https://doi.org/10.1609/aaai.v35i3.16312
AAAI Technical Track on Computer Vision II