Cross-Dataset Experimental Study of Radar-Camera Fusion in Bird’s-Eye View

Cross-Dataset Experimental Study of Radar-Camera Fusion in Bird’s-Eye View
Lukas Stephan Stäcker, Philipp Heidenreich, Jason Raphael Rambach, Didier Stricker
In: IEEE (Hrsg.). Proceedings of the 31st European Signal Processing Conference. European Signal Processing Conference (EUSIPCO-2023), 31st, September 4-8, Helsinki, Finland, IEEE, 2023.

Abstract:

By exploiting complementary sensor information, radar and camera fusion systems have the potential to provide a highly robust and reliable perception system for advanced driver assistance systems and automated driving functions. Recent advances in camera-based object detection offer new radar-camera fusion possibilities with bird’s eye view feature maps. In this work, we propose a novel and flexible fusion network and evaluate its performance on two datasets: nuScenes and View-of-Delft. Our experiments reveal that while the camera branch needs large and diverse training data, the radar branch benefits more from a high-performance radar. Using transfer learning, we improve the camera’s performance on the smaller dataset. Our results further demonstrate that the radar-camera fusion approach significantly outperforms the camera-only and radar-only baselines.