RDFM: Robust Deep Feature Matching for Multimodal Remote-Sensing Images

Robust feature matching for multimodal remote-sensing images remains challenging due to the significant nonlinear radiation difference (NRD) caused by modality variations. In this letter, we present a novel feature-matching method for multimodal remote-sensing images, called robust deep feature matching (RDFM), which exploits only deep features extracted by a pretrained Visual Geometry Group (VGG) network to achieve competitive performance. It is shown that template matching of these pretrained features is robust to NRD for various multimodal remote-sensing images, and no additional training is required to improve the matching performance. To extract as many correspondences as possible, we use dense template matching to obtain point correspondences and introduce a 4-D convolution-based implementation of dense template matching for the sake of computational efficiency. RDFM consists of two main steps. First, enormous coarse correspondences are extracted by applying dense template matching at the deep layer of the pretrained network, and then a coarse-to-fine hierarchical refinement is performed to obtain high-quality correspondences. To verify the effectiveness of RDFM, six different types of multimodal image datasets are used in our experiments, including day-night, depth-optical, infrared-optical, map-optical, optical-optical, and SAR-optical datasets. The comprehensive experimental results show that RDFM can overcome the problem caused by NRD and achieves a better performance than the state-of-the-art methods for multimodal remote-sensing image matching. The code of RDFM is publicly available at https://github.com/Fans2017/RDFM.

Automated summary

This letter presents a method called Robust Deep Feature Matching (RDFM) for robust feature matching in multimodal remote-sensing images. RDFM utilizes pretrained deep features extracted by a VGG network for template matching, and achieves competitive performance without additional training. Experimental results demonstrate the effectiveness of RDFM in overcoming nonlinear radiation difference (NRD) caused by modality variations.