Holistic Prototype Activation for Few-Shot Segmentation

Conventional deep CNN-based segmentation approaches have achieved satisfactory performance in recent years, however, they are essentially Big Data-driven technologies and are difficult to generalize to unseen categories. Few-shot segmentation is subsequently developed to perform pertinent operations in a low-data regime. Unfortunately, due to the training paradigm and network architecture factors, existing methods are prone to overfit the targets of base categories and yield inaccurate segmentation boundaries, which impedes the research progress to some extent. In this paper, we propose a Holistic Prototype Activation (HPA) network to alleviate these problems. Its novel designs can be summarized in three aspects: 1) A training-free scheme to derive the prior representations of base categories. 2) Prototype Activation Module (PAM) that generates reliable activation maps and well-matched query features by filtering the objects of irrelevant classes with high confidence. 3) Cross-Referenced Decoder (CRD) for interacted feature reweighting and multi-level feature aggregation. Extensive experiments on standard few-shot segmentation benchmarks (PASCAL-5(i) and COCO-20(i)) verify the effectiveness of our method. On top of that, the superior performance on multiple extended tasks, such as weak-label segmentation, zero-shot segmentation, and video object segmentation, also illustrates its flexibility and versatility. Our code is publicly available at https://github.com/chunbolang/HPA.

Automated summary

Conventional deep CNN-based segmentation approaches are difficult to generalize to unseen categories, so few-shot segmentation is developed to handle this issue. However, existing methods often overfit base categories and produce inaccurate segmentation boundaries. In this paper, a Holistic Prototype Activation (HPA) network is proposed to alleviate these problems by introducing novel designs, such as a training-free scheme, a Prototype Activation Module (PAM), and a Cross-Referenced Decoder (CRD). Experimental results on standard few-shot segmentation benchmarks and extended tasks demonstrate the effectiveness, flexibility, and versatility of the proposed method. The code is publicly available.