Keypoint Message Passing for Video-Based Person Re-identification


  • Di Chen Nanjing University of Science and Technology MPI Informatics
  • Andreas Doering University of Bonn
  • Shanshan Zhang Nanjing University of Science and Technology
  • Jian Yang Nanjing University of Science and Technology
  • Juergen Gall University of Bonn
  • Bernt Schiele MPI Informatics



Computer Vision (CV)


Video-based person re-identification~(re-ID) is an important technique in visual surveillance systems which aims to match video snippets of people captured by different cameras. Existing methods are mostly based on convolutional neural networks~(CNNs), whose building blocks either process local neighbor pixels at a time, or, when 3D convolutions are used to model temporal information, suffer from the misalignment problem caused by person movement. In this paper, we propose to overcome the limitations of normal convolutions with a human-oriented graph method. Specifically, features located at person joint keypoints are extracted and connected as a spatial-temporal graph. These keypoint features are then updated by message passing from their connected nodes with a graph convolutional network~(GCN). During training, the GCN can be attached to any CNN-based person re-ID model to assist representation learning on feature maps, whilst it can be dropped after training for better inference speed. Our method brings significant improvements over the CNN-based baseline model on the MARS dataset with generated person keypoints and a newly annotated dataset: PoseTrackReID. It also defines a new state-of-the-art method in terms of top-1 accuracy and mean average precision in comparison to prior works.




How to Cite

Chen, D., Doering, A., Zhang, S., Yang, J., Gall, J., & Schiele, B. (2022). Keypoint Message Passing for Video-Based Person Re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, 36(1), 239-247.



AAAI Technical Track on Computer Vision I