RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving

Authors

  • Peixuan Li Shenyang Institute of Automation,Chinese Academy of Sciences,Shenyang 11016, China; Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China; University of Chinese Academy of Sciences, Beijing 100049, China; Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences; Key Lab of Image Understanding and Computer Vision, Liaoning Province
  • Shun Su Shenyang Institute of Automation,Chinese Academy of Sciences,Shenyang 11016, China; Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China; University of Chinese Academy of Sciences, Beijing 100049, China; State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China
  • Huaici Zhao Shenyang Institute of Automation,Chinese Academy of Sciences,Shenyang 11016, China; Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China; Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences; Key Lab of Image Understanding and Computer Vision, Liaoning Province

Keywords:

3D Computer Vision, Vision for Robotics & Autonomous Driving

Abstract

Although the recent image-based 3D object detection methods using Pseudo-LiDAR representation have shown great capabilities, a notable gap in efficiency and accuracy still exist compared with LiDAR-based methods. Besides, over-reliance on the stand-alone depth estimator, requiring a large number of pixel-wise annotations in the training stage and more computation in the inferencing stage, limits the scaling application in the real world. In this paper, we propose an efficient and accurate 3D object detection method from stereo images, named RTS3D. Different from the 3D occupancy space in the Pseudo-LiDAR similar methods, we design a novel 4D feature-consistent embedding (FCE) space as the intermediate representation of the 3D scene without depth supervision. The FCE space encodes the object's structural and semantic information by exploring the multi-scale feature consistency warped from stereo pair. Furthermore, a semantic-guided RBF (Radial Basis Function) and a structure-aware attention module are devised to reduce the influence of FCE space noise without instance mask supervision. Experiments on KITTI benchmark show that RTS3D is the first true real-time system (FPS>24) for stereo image 3D detection meanwhile achieves 10% improvement in average precision comparing with the previous state-of-the-art method.

Downloads

Published

2021-05-18

How to Cite

Li, P., Su, S., & Zhao, H. (2021). RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3), 1930-1939. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16288

Issue

Section

AAAI Technical Track on Computer Vision II