Full-Velocity Radar Returns by Radar-Camera Fusion

A distinctive feature of Doppler radar is the measurement of velocity in the radial direction for radar points. However, the missing tangential velocity component hampers object velocity estimation as well as temporal integration of radar sweeps in dynamic scenes. Recognizing that fusing camera with radar provides complementary information to radar, in this paper we present a closed-form solution for the point-wise, full-velocity estimate of Doppler returns using the corresponding optical flow from camera images. Additionally, we address the association problem between radar returns and camera images with a neural network that is trained to estimate radar-camera correspondences. Experimental results on the nuScenes dataset verify the validity of the method and show significant improvements over the state-of-the-art in velocity estimation and accumulation of radar points.

Automated summary

This paper presents a closed-form solution for the full-velocity estimate of Doppler returns using optical flow from camera images, and addresses the association problem between radar returns and camera images with a trained neural network. Experimental results on the nuScenes dataset validate the effectiveness of the method in velocity estimation and accumulation of radar points, showing significant improvements over the state-of-the-art.