Journal
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
Volume 60, Issue -, Pages -Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TGRS.2022.3165209
Keywords
Image segmentation; Buildings; Radar polarimetry; Optical imaging; Optical sensors; Adaptive optics; Synthetic aperture radar; Building extraction; deep learning; knowledge distillation; missing modality; semantic segmentation; synthetic aperture radar (SAR); transfer learning
Categories
Funding
- Natural Science Foundation of China [62101371, 62076241]
- Jiangsu Province Science Foundation for Youths [BK20210707]
- Ministry of Science, Innovation and Universities of Spain [RTI2018-098651-B-C54, PID2019-110315RB-I00]
- Valencian Government of Spain [GV/2020/167]
- European Fund for Economic and Regional Development (FEDER)-Junta de Extremadura [GR18060]
- European Union through the H2020 EOXPOSURE Project [734541]
Ask authors/readers for more resources
Synthetic aperture radar (SAR) images have been widely used in intelligent remote sensing (RS) image interpretation, especially under poor weather conditions. However, compared to optical images, SAR images have less rich and interpretable semantics. In this article, a novel method based on the DisOptNet network is proposed to improve segmentation performance through knowledge transfer from optical images. Experimental results demonstrate the effectiveness of DisOptNet in generating building footprints under real scenarios.
Synthetic aperture radar (SAR) images provide all-weather and all-time capabilities for Earth observation, which becomes highly beneficial in the field of intelligent remote sensing (RS) image interpretation. Due to these advantages, SAR images have been widely exploited in automatic building segmentation tasks under poor weather conditions, especially when disasters happen. However, compared to optical images, the semantics inherent to SAR images are less rich and interpretable due to factors such as speckle noise and imaging geometry. In this scenario, most state-of-the-art methods are focused on designing advanced network architectures or loss functions for building footprint extraction. However, few works have been oriented toward improving segmentation performance through knowledge transfer from optical images. In this article, we propose a novel method based on the DisOptNet network, which can distill the useful semantic knowledge from optical images into a network only trained with SAR data. Specifically, we first analyze the multilevel feature discrepancies between multiple stages of the networks pretrained on the two image modalities. We observe that feature discrepancies start to increase as the encoding stage gradually changes from low level to high level. Based on such observation, we reuse the early stage features and construct parallel convolutional neural network (CNN) branches that are responsible for capturing high-level domain-specific knowledge for each image modality. The optical branch is aimed at mimicking feature generation at the optical pretrained network given the input SAR images. Then, an aggregation module is introduced to calibrate and fuse the features from different modalities while generating the building segments. Extensive experiments were conducted on a large-scale multisensor all-weather building segmentation dataset with state-of-the-art methods used for comparison. Our experimental results validate the effectiveness of DisOptNet, which demonstrates great potential in the task of weather-independent building footprint generation under real scenarios. The codes of this article will be made publicly available at https://github.com/jiankang1991/TGRS_DisOptNet.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available