From Image Transfer to Object Transfer: Cross-Domain Instance Segmentation Based on Center Point Feature Alignment

Remote sensing images can have significant appearance differences due to various factors, such as atmospheric conditions, sensor types, seasons, and capture times. Therefore, when applying a pretrained instance segmentation deep learning model to newly accessed remote sensing images, the model's performance tends to decrease significantly. Current mainstream image-based or feature-based domain adaptation methods are not designed specifically for the cross-domain instance segmentation problem. These methods attempt to align the whole images, which may not be optimal for instance segmentation tasks. To address this issue, we propose a cross-domain instance segmentation method based on object-level alignment. Instead of aligning the entire images from both datasets, we only align the features of each object instance, particularly the representative center point features. Our approach mainly consists of an improved contour-based instance segmentation model for object-based domain adaptation, an object-pasting enhancement technique based on Fourier domain adaptation (FDA) that effectively reduces the gap between the source and target domains of the object instances, and a self-training strategy that dynamically generates pseudolabels for iterative model training. Our experiments on cross-domain building instance segmentation demonstrate that the proposed method achieves a 9.5 intersection over union (IoU) improvement over the current best method. Additionally, experiments on a cross-domain close-range dataset involving transfer between simulated and real street images show that our method significantly outperforms the current best method by 6.5 mean average precision (mAP). These results on remote sensing and close-range datasets validate the universality of our approach.

Automated summary

Remote sensing images can vary significantly due to factors like atmospheric conditions and sensor types, making it challenging to apply pretrained instance segmentation models to new images. Existing domain adaptation methods are not tailored for cross-domain instance segmentation, which requires aligning object-level features instead of whole images. To address this, we propose a method based on object-level alignment, incorporating an improved contour-based instance segmentation model, a Fourier domain adaptation technique for object-pasting enhancement, and a self-training strategy. Experimental results on different datasets demonstrate the effectiveness and universality of our approach, achieving substantial improvements in intersection over union (IoU) and mean average precision (mAP) compared to current state-of-the-art methods.