CROCO: CROSS-MODAL CONTRASTIVE LEARNING FOR LOCALIZATION OF EARTH OBSERVATION DATA

It is of interest to localize a ground-based LiDAR point cloud on remote sensing imagery. In this work, we tackle a subtask of this problem, i.e. to map a digital elevation model (DEM) rasterized from aerial LiDAR point cloud on the aerial imagery. We proposed a contrastive learning-based method that trains on DEM and high-resolution optical imagery and experiment the framework on different data sampling strategies and hyperparameters. In the best scenario, the Top-1 score of 0.71 and Top-5 score of 0.81 are obtained. The proposed method is promising for feature learning from RGB and DEM for localization and is potentially applicable to other data sources too. Source code is released at https://github.com/wtseng530/AVLocalization.

Automated summary

This paper investigates the localization of ground-based LiDAR point cloud on remote sensing imagery. A contrastive learning-based method is proposed, which trains on a digital elevation model (DEM) and high-resolution optical imagery. Experimental results show that the method achieves high scores and has the potential for feature learning and localization.