4.7 Article

Panoptic Segmentation Meets Remote Sensing

Journal

REMOTE SENSING
Volume 14, Issue 4, Pages -

Publisher

MDPI
DOI: 10.3390/rs14040965

Keywords

deep learning; aerial image; dataset; semantic segmentation; instance segmentation; panoptic segmentation

Funding

  1. Conselho Nacional de Pesquisa e Desenvolvimento [434838/2018-7, 305769/2017-0]
  2. Coordenacao de Aperfeicoamento de Pessoal de Nivel Superior [001]
  3. University of Brasilia

Ask authors/readers for more resources

Panoptic segmentation has great potential in remotely sensed data as it combines instance and semantic predictions to detect countable objects and different backgrounds simultaneously. However, challenges such as labeling large images, generating DL samples in the panoptic segmentation format, handling large remote sensing images, and software compatibility issues have hindered the growth of this task. This study addresses these challenges by providing a pipeline for generating panoptic segmentation datasets, software for creating deep learning samples in the COCO annotation format, a novel dataset, compatibility with remote sensing data using Detectron2 software, and evaluation on the urban setting.
Panoptic segmentation combines instance and semantic predictions, allowing the detection of countable objects and different backgrounds simultaneously. Effectively approaching panoptic segmentation in remotely sensed data is very promising since it provides a complete classification, especially in areas with many elements as the urban setting. However, some difficulties have prevented the growth of this task: (a) it is very laborious to label large images with many classes, (b) there is no software for generating DL samples in the panoptic segmentation format, (c) remote sensing images are often very large requiring methods for selecting and generating samples, and (d) most available software is not friendly to remote sensing data formats (e.g., TIFF). Thus, this study aims to increase the operability of panoptic segmentation in remote sensing by providing: (1) a pipeline for generating panoptic segmentation datasets, (2) software to create deep learning samples in the Common Objects in Context (COCO) annotation format automatically, (3) a novel dataset, (4) leverage the Detectron2 software for compatibility with remote sensing data, and (5) evaluate this task on the urban setting. The proposed pipeline considers three inputs (original image, semantic image, and panoptic image), and our software uses these inputs alongside point shapefiles to automatically generate samples in the COCO annotation format. We generated 3400 samples with 512 x 512 pixel dimensions and evaluated the dataset using Panoptic-FPN. Besides, the metric analysis considered semantic, instance, and panoptic metrics, obtaining 93.865 mean intersection over union (mIoU), 47.691 Average (AP) Precision, and 64.979 Panoptic Quality (PQ). Our study presents the first effective pipeline for generating panoptic segmentation data for remote sensing targets.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available