Learning panoptic segmentation through feature discriminability

Panoptic segmentation has attracted increasing attention as a joint task of semantic and instance segmentation. However, previous works have not noticed that the different requirements for semantic and instance segmentation can lead to conflict of feature discriminability. Instance segmentation mainly focuses on the central area of each instance in things regions, while semantic segmentation focuses on the whole region of a specific class. To resolve it, we propose: 1) a Dual-FPN framework which separates the shared Feature Pyramid Network (FPN) in previous works to reduce the conflict of receptive field and meet different requirements of the two tasks; 2) a Region Refinement Module which leverages the prediction of semantic segmentation to refine the result of instance segmentation and resolves the conflict between the things regions and the stuff regions. Experimental results on Cityscapes dataset and Mapillary Vistas dataset show that our proposed method can improve the result of both things and stuff and obtain state-of-the-art performance. (c) 2021 Published by Elsevier Ltd.

Automated summary

Panoptic segmentation is a joint task of semantic and instance segmentation, and conflicting feature discriminability may arise due to different requirements. To address this issue, a Dual-FPN framework and a Region Refinement Module are proposed, achieving state-of-the-art performance on Cityscapes and Mapillary Vistas datasets.