A self-attention capsule feature pyramid network for water body extraction from remote sensing imagery

Timely and accurately measuring surface water bodies and monitoring their conditions and changes are greatly important to a wide range of environmental and social activities. Recently, with the development of optical remote sensing sensors in resolutions and qualities, as well as the convenience in data acquisition, remote sensing images have become an important data source for assisting water body measurements. However, due to the considerable variations of water bodies in shapes, areas, and sizes, the diversities of colour appearances, and the complicated surface and surrounding scenarios, it is still challenging to automatically and accurately extract water bodies from remote sensing images. In this paper, we develop a novel self-attention capsule feature pyramid network (SA-CapsFPN) to extract water bodies from remote sensing images. By designing a deep capsule feature pyramid architecture, the SA-CapsFPN can extract and fuse multi-level and multiscale high-order capsule features to provide a high-resolution, semantically strong feature encoding for improving pixel-wise water body extraction accuracy. With the integration of the context-augmentation and self-attention modules, the SA-CapsFPN can exploit multiscale contextual properties and emphasize channel-wise informative features, thereby enhancing the feature representation capability. The SA-CapsFPN performs superiorly in extracting water bodies of varying shapes, areas, and sizes, as well as diverse surface and environmental scenarios. Quantitative evaluations on two big remote sensing image datasets show that an overall performance with a P, an R, and an F (score) of 0.9771, 0.9684, and 0.9727, respectively, are achieved. Comparative studies with five deep learning based methods also demonstrate the applicability and superiority of the SA-CapsFPN in water body extraction tasks.

Automated summary

This paper introduces a novel self-attention capsule feature pyramid network (SA-CapsFPN) for extracting water bodies from remote sensing images. By designing a deep capsule feature pyramid architecture to extract and fuse multi-level and multiscale high-order capsule features, the SA-CapsFPN provides high-resolution, semantically strong feature encoding to improve pixel-wise water body extraction accuracy. The SA-CapsFPN performs well in extracting water bodies of varying shapes, areas, and sizes, as well as diverse surface and environmental scenarios.