Cross-Image Region Mining With Region Prototypical Network for Weakly Supervised Segmentation

Weakly supervised image segmentation trained with image-level labels usually suffers from inaccurate coverage of object areas during the generation of the pseudo groundtruth. This is because the object activation maps are trained with the classification objective and lack the ability to generalize. To improve the generality of the object activation maps, we propose a region prototypical network (RPNet) to explore the cross-image object diversity of the training set. Similar object parts across images are identified via region feature comparison. Object confidence is propagated between regions to discover new object areas while background regions are suppressed. Experiments show that the proposed method generates more complete and accurate pseudo object masks while achieving state-of-the-art performance on PASCAL VOC 2012 and MS COCO. In addition, we investigate the robustness of the proposed method on reduced training sets. The code is available at https://github.com/liuweide01/RPNet-Weakly-Supervised-Segmentation.

Automated summary

Weakly supervised image segmentation trained with image-level labels usually suffer from inaccurate coverage of object areas during pseudo groundtruth generation. To enhance the generality of object activation maps, we propose a region prototypical network (RPNet) to explore cross-image object diversity. Similar object parts are identified through region feature comparison, with object confidence propagated to discover new object areas and suppress background regions. Experiments demonstrate that our approach generates more complete and accurate pseudo object masks and achieves state-of-the-art performance on PASCAL VOC 2012 and MS COCO. We also investigate the robustness of our method on reduced training sets. The code can be accessed at https://github.com/liuweide01/RPNet-Weakly-Supervised-Segmentation.