RADIANT: Radar-Image Association Network for 3D Object Detection

Authors

  • Yunfei Long Michigan State University
  • Abhinav Kumar Michigan State University
  • Daniel Morris Michigan State University
  • Xiaoming Liu Michigan State University
  • Marcos Castro Ford Motor Company
  • Punarjay Chakravarty Ford Motor Company

DOI:

https://doi.org/10.1609/aaai.v37i2.25270

Keywords:

CV: Object Detection & Categorization, CV: Vision for Robotics & Autonomous Driving, CV: 3D Computer Vision

Abstract

As a direct depth sensor, radar holds promise as a tool to improve monocular 3D object detection, which suffers from depth errors, due in part to the depth-scale ambiguity. On the other hand, leveraging radar depths is hampered by difficulties in precisely associating radar returns with 3D estimates from monocular methods, effectively erasing its benefits. This paper proposes a fusion network that addresses this radar-camera association challenge. We train our network to predict the 3D offsets between radar returns and object centers, enabling radar depths to enhance the accuracy of 3D monocular detection. By using parallel radar and camera backbones, our network fuses information at both the feature level and detection level, while at the same time leveraging a state-of-the-art monocular detection technique without retraining it. Experimental results show significant improvement in mean average precision and translation error on the nuScenes dataset over monocular counterparts. Our source code is available at https://github.com/longyunf/radiant.

Downloads

Published

2023-06-26

How to Cite

Long, Y., Kumar, A., Morris, D., Liu, X., Castro, M., & Chakravarty, P. (2023). RADIANT: Radar-Image Association Network for 3D Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 37(2), 1808-1816. https://doi.org/10.1609/aaai.v37i2.25270

Issue

Section

AAAI Technical Track on Computer Vision II