A Cross-Modality Feature Transfer Method for Target Detection in SAR Images

Synthetic aperture radar (SAR) ship detection methods have achieved remarkable progress in recent years. However, unlike RGB images, the characteristics of SAR imaging will result in non-intuitive feature representations. Furthermore, due to the insufficient data of SAR images, existing methods relying on plenty of labeled SAR images may be hard to achieve promising performance. To address the aforementioned issues, a cross-modality feature transfer (CMFT) method is proposed in this article, which enhances feature representations in the SAR modality by transferring rich knowledge in the RGB modality. First, we propose a multilevel modality alignment network (MMAN), which encourages the model to effectively learn modality-invariant features and alleviate the large cross-modality discrepancies by aligning features from multilevels (scene level, local level, global level, and instance level). Second, to address the underperformance of samples with non-intuitive features in the modality alignment, we introduce a hard-sample supervision module (HSM) in the stage of feature extraction, which can thoroughly exploit the feature of hard-to-align samples by giving more optimization energy for them. Third, to enhance the discriminability of instance-level features, a feature complementary module (FCM) is customized to fully explore the potential complementary clues between instance-level features and context information for the instance-level feature alignment. Extensive experimental results demonstrate that the CMFT outperforms the state-of-the-art detectors. Compared to the baseline model, CMFT improves the accuracy by 3.1% mean average precision (mAP) on the SSDD dataset and 3.4% mAP on the HRSID dataset, demonstrating its superior SAR ship detection performance.

Automated summary

This article proposes a cross-modality feature transfer (CMFT) method to enhance feature representations in SAR modality by transferring knowledge from RGB modality. Through a multilevel modality alignment network (MMAN), a hard-sample supervision module (HSM), and a feature complementary module (FCM), the non-intuitive feature representations in SAR imaging are addressed. Experimental results demonstrate the superiority of CMFT method in SAR ship detection performance.