Depth Privileged Object Detection in Indoor Scenes via Deformation Hallucination


  • Zhijie Zhang Shanghai Jiao Tong University
  • Yan Liu Shanghai Jiao Tong University
  • Junjie Chen Shanghai Jiao Tong University
  • Li Niu Shanghai Jiao Tong University
  • Liqing Zhang Shanghai Jiao Tong University



Object Detection & Categorization, Transfer/Adaptation/Multi-task/Meta/Automated Learning, Multi-modal Vision


RGB-D object detection has achieved significant advance, because depth provides complementary geometric information to RGB images. Considering depth images are unavailable in some scenarios, we focus on depth privileged object detection in indoor scenes, where the depth images are only available in the training phase. Under this setting, one prevalent research line is modality hallucination, in which depth image and depth feature are the common choices for hallucinating. In contrast, we choose to hallucinate depth deformation, which is explicit geometric information and efficient to hallucinate. Specifically, we employ the deformable convolution layer with augmented offsets as our deformation module and regard the offsets as geometric deformation, because the offsets enable flexibly sampling over the object and transforming to a canonical shape for ease of detection. In addition, we design a quality-based mechanism to avoid negative transfer of depth deformation. Experimental results and analyses on NYUDv2 and SUN RGB-D demonstrate the effectiveness of our method against the state-of-the-art methods for depth privileged object detection.




How to Cite

Zhang, Z., Liu, Y., Chen, J., Niu, L., & Zhang, L. (2021). Depth Privileged Object Detection in Indoor Scenes via Deformation Hallucination. Proceedings of the AAAI Conference on Artificial Intelligence, 35(4), 3456-3464.



AAAI Technical Track on Computer Vision III