Fusion-Based Correlation Learning Model for Cross-Modal Remote Sensing Image Retrieval

With the increasing of cross-modal data, cross-modal retrieval has attracted more attention in remote sensing (RS), since it provides a more flexible and convenient way to obtain interesting information than traditional retrieval. However, existing methods cannot fully exploit the semantic information, which only focuses on the semantic consistency, and ignore the information complementarity between different modalities. In this letter, to bridge the modality gap, we propose a novel fusion-based correlation learning model (FCLM) for image-text retrieval in RS. Specifically, a cross-modal-fusion network is designed to capture the intermodality complementary information and fused feature. The fused knowledge is furtherly transferred to supervise the learning of modality-specific network by knowledge distillation, which is helpful in improving the discriminative ability of feature representation and enhancing the intermodality semantic consistency to solve the heterogeneity gap problem. Finally, extensive experiments have been conducted on a public dataset and experimental results have shown that the FCLM method is effective in performing cross-modal retrieval and outperforms several baseline methods.

Automated summary

In this study, a fusion-based correlation learning model is proposed to address the heterogeneity gap in remote sensing image-text retrieval. By designing a cross-modal fusion network and utilizing knowledge distillation, this model improves the discriminative ability of feature representation and enhances the intermodality semantic consistency.