Behind the Curtain: Learning Occluded Shapes for 3D Object Detection


  • Qiangeng Xu University of Southern California
  • Yiqi Zhong University of Southern California
  • Ulrich Neumann University of Southern California



Computer Vision (CV), Cognitive Modeling & Cognitive Systems (CMS), Machine Learning (ML), Domain(s) Of Application (APP)


Advances in LiDAR sensors provide rich 3D data that supports 3D scene understanding. However, due to occlusion and signal miss, LiDAR point clouds are in practice 2.5D as they cover only partial underlying shapes, which poses a fundamental challenge to 3D perception. To tackle the challenge, we present a novel LiDAR-based 3D object detection model, dubbed Behind the Curtain Detector (BtcDet), which learns the object shape priors and estimates the complete object shapes that are partially occluded (curtained) in point clouds. BtcDet first identifies the regions that are affected by occlusion and signal miss. In these regions, our model predicts the probability of occupancy that indicates if a region contains object shapes and integrates this probability map with detection features and generates high-quality 3D proposals. Finally, the occupancy estimation is integrated into the proposal refinement module to generate accurate bounding boxes. Extensive experiments on the KITTI Dataset and the Waymo Open Dataset demonstrate the effectiveness of BtcDet. Particularly for the 3D detection of both cars and cyclists on the KITTI benchmark, BtcDet surpasses all of the published state-of-the-art methods by remarkable margins. Code is released.




How to Cite

Xu, Q., Zhong, Y., & Neumann, U. (2022). Behind the Curtain: Learning Occluded Shapes for 3D Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 36(3), 2893-2901.



AAAI Technical Track on Computer Vision III