Domain Adaptation for Remote Sensing Image Semantic Segmentation: An Integrated Approach of Contrastive Learning and Adversarial Learning

Although semantic segmentation models based on deep neural networks (DNNs) have achieved excellent results, generalizing well from one remote sensing dataset (source domain) to another dataset with different acquisition conditions (target domain) remains a major challenge. Many domain adaptation (DA) approaches have been proposed to address this problem. DA aims to help DNNs learn a generalizable representation space in which source and target domains have similar feature distributions, but most of the existing DA approaches have difficulty in aligning the high-dimensional image representations of two domains directly. In this study, we proposed a model integrating contrastive learning and adversarial learning in a unified framework for aligning two domains in both representation space and spatial layout. Specifically, the model consists of a semantic segmentation network for feature extraction and two branches for DA. The first branch is used for adaptation in representation space directly by a proposed pixelwise contrastive loss, while the second branch is used for adaptation in predicted results to help two domains have similar spatial layouts through a novel but simple entropy-based similarity discriminator. Additionally, a training strategy called category similarity matching sampling was proposed to provide source and target image pairs with similar category composition for each training iteration, which can help the two branches work better. Extensive experiments indicated that the two branches can benefit each other to gain a superior performance and DA pretraining by our methods can achieve impressive results with only a small number of target labeled samples.

Automated summary

This study proposes a model that integrates contrastive learning and adversarial learning to align two remote sensing datasets with different acquisition conditions in both representation space and spatial layout. It achieves adaptive alignment at the pixel level and the predicted results level using a semantic segmentation network and two domain adaptation branches. Additionally, a training strategy called category similarity matching sampling is proposed to improve the model's performance by providing similar category compositions in image pairs.