Salient-VPR: Salient Weighted Global Descriptor for Visual Place Recognition

Visual place recognition (VPR) is a widely investigated but challenging problem, which must deal with the twin problems of changed appearance caused by varying weather conditions, illumination, and seasons, as well as dynamic objects in complex environments. In this article, we proposed salient VPR by combining image retrieval, semantic information, and saliency cues to achieve accurate estimations. We provide a novel formulation for combining the local semantic features into global descriptors. Unlike traditional global descriptors that typically summarize the whole visual content in the images or image regions, we aggregate dense local semantic descriptors based on pixel-level semantic scores to form global semantic descriptors. Then we calculate predicted saliency descriptors to learn the representation of static objects from images. These salient descriptions are learned from a dataset tailored for VPR missions. Finally, we introduce a late-fusion module, which increases the stability of the descriptor and avoids performance degradation caused by the limitations of semantic and saliency prediction. Our method outperforms traditional global descriptors in experiments, attaining state-of-the-art visual location recognition scores on a variety of challenging datasets such as Pitts30 k, Mapillary Street-level Sequences (MSLS), and Tokyo24/7.

Automated summary

This article proposes a salient visual place recognition method that combines image retrieval, semantic information, and saliency cues for accurate estimations. The method uses a novel formulation to combine local semantic features into global descriptors and predicts saliency descriptors to learn the representation of static objects. A late-fusion module is introduced to increase the stability of the descriptor.