Depth Privileged Object Detection in Indoor Scenes via Deformation Hallucination

Authors

  • Zhijie Zhang Shanghai Jiao Tong University
  • Yan Liu Shanghai Jiao Tong University
  • Junjie Chen Shanghai Jiao Tong University
  • Li Niu Shanghai Jiao Tong University
  • Liqing Zhang Shanghai Jiao Tong University

Keywords:

Object Detection & Categorization, Transfer/Adaptation/Multi-task/Meta/Automated Learning, Multi-modal Vision

Abstract

RGB-D object detection has achieved significant advance, because depth provides complementary geometric information to RGB images. Considering depth images are unavailable in some scenarios, we focus on depth privileged object detection in indoor scenes, where the depth images are only available in the training phase. Under this setting, one prevalent research line is modality hallucination, in which depth image and depth feature are the common choices for hallucinating. In contrast, we choose to hallucinate depth deformation, which is explicit geometric information and efficient to hallucinate. Specifically, we employ the deformable convolution layer with augmented offsets as our deformation module and regard the offsets as geometric deformation, because the offsets enable flexibly sampling over the object and transforming to a canonical shape for ease of detection. In addition, we design a quality-based mechanism to avoid negative transfer of depth deformation. Experimental results and analyses on NYUDv2 and SUN RGB-D demonstrate the effectiveness of our method against the state-of-the-art methods for depth privileged object detection.

Downloads

Published

2021-05-18

How to Cite

Zhang, Z., Liu, Y., Chen, J., Niu, L., & Zhang, L. (2021). Depth Privileged Object Detection in Indoor Scenes via Deformation Hallucination. Proceedings of the AAAI Conference on Artificial Intelligence, 35(4), 3456-3464. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16459

Issue

Section

AAAI Technical Track on Computer Vision III