CMLocate: A cross-modal automatic visual geo-localization framework for a natural environment without GNSS information

In this paper, a new approach to visual geo-localization for natural environments is proposed. The digital elevation model (DEM) data in virtual space is rendered and construct a panoramic skyline database is constructed. By combining the skyline database with real-world image data (used as the queries to be localized), visual geo-localization is treated as a cross-modal image retrieval problem for panoramic skyline images, creating a unique new visual geo-localization benchmark for the natural environment. Specifically, the semantic segmentation model named LineNet is proposed, for skyline extractions from query images, which has proven to be robust to a variety of complex natural environments. On the aforementioned benchmarks, the fully automatic method is elaborated for large-scale cross-modal localization using panoramic skyline images. Finally, the compound index is delicately designed to reduce the storage space of the positioning global descriptors and improve the retrieval efficiency. Moreover, the proposed method is proven to outperform most state-of-the-art methods.

Automated summary

In this paper, a new approach to visual geo-localization for natural environments is proposed by creating a panoramic skyline database using digital elevation model (DEM) data in virtual space. The combination of the skyline database and real-world image data enables visual geo-localization as a cross-modal image retrieval problem. The paper introduces the LineNet semantic segmentation model for skyline extractions, which has proven to be robust in complex natural environments. Additionally, a compound index is designed to reduce storage space and improve retrieval efficiency, resulting in the proposed method outperforming most state-of-the-art methods.