Semantic Image Alignment for Vehicle Localization

Accurate and reliable localization is a fundamental requirement for autonomous vehicles to use map information in higher-level tasks such as navigation or planning. In this paper, we present a novel approach to vehicle localization in dense semantic maps, including vectorized high-definition maps or 3D meshes, using semantic segmentation from a monocular camera. We formulate the localization task as a direct image alignment problem on semantic images, which allows our approach to robustly track the vehicle pose in semantically labeled maps by aligning virtual camera views rendered from the map to sequences of semantically segmented camera images. In contrast to existing visual localization approaches, the system does not require additional keypoint features, handcrafted localization landmark extractors or expensive LiDAR sensors. We demonstrate the wide applicability of our method on a diverse set of semantic mesh maps generated from stereo or LiDAR as well as manually annotated HD maps and show that it achieves reliable and accurate localization in real-time.

Automated summary

The paper presents a novel approach to vehicle localization in dense semantic maps using semantic segmentation, which addresses the fundamental requirement of accurate and reliable localization for autonomous vehicles to utilize map information in higher-level tasks. By formulating the localization task as a direct image alignment problem on semantic images, the approach achieves robust tracking of vehicle pose in semantically labeled maps without the need for additional keypoint features or expensive LiDAR sensors, demonstrating wide applicability and real-time performance.