SIFNet: A self-attention interaction fusion network for multisource satellite imagery template matching

Multisource satellite images provide abundant and complementary earth observations, while nonlinear radio-metric and geometric distortions (such as scale and rotation variations) between these multimodal images pose remarkable challenges for further remote sensing applications, such as change detection. We therefore proposed a template matching algorithm based on self-attention interactive fusion network, named SIFNet, to align multisource satellite images. First, a feature pyramid network was first conducted to extract multiscale features for each pixel, with the template and reference images as the inputs. Then, the extracted features were fused by self-attention layers in Transformer for information interaction. Third, the similarity and semantic matching loss functions were developed to convert satellite imagery registration into regression task, allowing SIFNet aligning multimodal patch images more efficiently based on point-to-point correspondence, instead of globally searching extremums as previous matching strategies did. We performed experiments based on four multimodal datasets (i. e. Google, GF-2, Landsat-8 and optical-SAR images) with various scenes to evaluate the performance and robustness of SIFNet. The results demonstrate the proposed SIFNet performed a comparable accuracy for tem-plate matching with other algorithms and was robust to geometric distortions and radiometric variations of multisource remote sensing data.

Automated summary

Multisource satellite images provide abundant and complementary earth observations, but the nonlinear distortions between these multimodal images pose challenges for remote sensing applications. We propose a template matching algorithm called SIFNet based on self-attention interactive fusion network to align these images. Experimental results show that SIFNet achieves comparable accuracy in template matching and is robust to geometric distortions and radiometric variations of multisource remote sensing data.