Self Supervised Progressive Network for High Performance Video Object Segmentation

Recently, self-supervised video object segmentation (VOS) has attracted much interest. However, most proxy tasks are proposed to train only a single backbone, which relies on a point-to-point correspondence strategy to propagate masks through a video sequence. Due to its simple pipeline, the performance of the single backbone paradigm is still unsatisfactory. Instead of following the previous literature, we propose our self-supervised progressive network (SSPNet) which consists of a memory retrieval module (MRM) and collaborative refinement module (CRM). The MRM can perform point-to-point correspondence and produce a propagated coarse mask for a query frame through self-supervised pixel-level and frame-level similarity learning. The CRM, which is trained via cycle consistency region tracking, aggregates the reference & query information and learns the collaborative relationship among them implicitly to refine the coarse mask. Furthermore, to learn semantic knowledge from unlabeled data, we also design two novel mask-generation strategies to provide the training data with meaningful semantic information for the CRM. Extensive experiments conducted on DAVIS-17, YouTube-VOS and SegTrack v2 demonstrate that our method surpasses the state-of-the-art self-supervised methods and narrows the gap with the fully supervised methods.

Automated summary

This paper proposes a self-supervised progressive network (SSPNet) for video object segmentation. SSPNet consists of a memory retrieval module (MRM) and a collaborative refinement module (CRM) to improve the performance. The MRM generates propagated coarse masks through self-supervised pixel-level and frame-level similarity learning, while the CRM refines the masks through cycle consistency region tracking. Novel mask-generation strategies are also designed to incorporate meaningful semantic information. Experimental results on multiple datasets demonstrate the superiority of SSPNet over state-of-the-art self-supervised methods, narrowing the gap with fully supervised methods.