Semantic Segmentation Network Using Local Relationship Upsampling for Remote Sensing Images

Semantic segmentation is a fundamental task in remote sensing image processing. It provides pixel-level classification, which is important for many applications, such as building extraction and land use mapping. The development of convolutional neural network has considerably improved the performance of semantic segmentation. Most semantic segmentation networks are the encoder-decoder structure. Bilinear interpolation is an ordinary upsampling method in the decoder, but bilinear interpolation only considers its own features and inserts three times its own features. This over-simple and data-independent bilinear upsampling may lead to suboptimal results. In this work, we propose an upsampling method based on local relations to replace bilinear interpolation. Upsampling is performed by correlating the local relationship of feature maps of adjacent stages, which can better integrate local and global information. We also design a fusion module based on local similarity. Our proposed method with ResNet101 as the backbone of the segmentation network can improve the average F-1 score and overall accuracy of the Vaihingen data set by 2.69% and 1.31%, respectively. Our proposed method also has fewer parameters and less inference time.

Automated summary

In this study, a novel upsampling method based on local relations was proposed to replace traditional bilinear interpolation for semantic segmentation, improving the integration of local and global information. By using ResNet101 as the backbone network, our proposed method achieved a 2.69% increase in average F-1 score and a 1.31% increase in overall accuracy on the Vaihingen dataset, with fewer parameters and shorter inference time.