Contrastive Multi-Task Dense Prediction


  • Siwei Yang Key Laboratory of Embedded System and Service Computing, Tongji University Hong Kong University of Science and Technology
  • Hanrong Ye Hong Kong University of Science and Technology
  • Dan Xu Hong Kong University of Science and Technology



CV: Scene Analysis & Understanding


This paper targets the problem of multi-task dense prediction which aims to achieve simultaneous learning and inference on a bunch of multiple dense prediction tasks in a single framework. A core objective in design is how to effectively model cross-task interactions to achieve a comprehensive improvement on different tasks based on their inherent complementarity and consistency. Existing works typically design extra expensive distillation modules to perform explicit interaction computations among different task-specific features in both training and inference, bringing difficulty in adaptation for different task sets, and reducing efficiency due to clearly increased size of multi-task models. In contrast, we introduce feature-wise contrastive consistency into modeling the cross-task interactions for multi-task dense prediction. We propose a novel multi-task contrastive regularization method based on the consistency to effectively boost the representation learning of the different sub-tasks, which can also be easily generalized to different multi-task dense prediction frameworks, and costs no additional computation in the inference. Extensive experiments on two challenging datasets (i.e. NYUD-v2 and Pascal-Context) clearly demonstrate the superiority of the proposed multi-task contrastive learning approach for dense predictions, establishing new state-of-the-art performances.




How to Cite

Yang, S., Ye, H., & Xu, D. (2023). Contrastive Multi-Task Dense Prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 37(3), 3190-3197.



AAAI Technical Track on Computer Vision III