Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision

Instance segmentation is an active topic in computer vision that is usually solved by using supervised learning approaches over very large datasets composed of object level masks. Obtaining such a dataset for any new domain can be very expensive and time-consuming. In addition, models trained on certain annotated categories do not generalize well to unseen objects. The goal of this paper is to propose a method that can perform unsupervised discovery of long-tail categories in instance segmentation, through learning instance embeddings of masked regions. Leveraging rich relationship and hierarchical structure between objects in the images, we propose self-supervised losses for learning mask embeddings. Trained on COCO [34] dataset without additional annotations of the long-tail objects, our model is able to discover novel and more fine-grained objects than the common categories in COCO. We show that the model achieves competitive quantitative results on LVIS [17] as compared to the supervised and partially supervised methods.

Automated summary

This research aims to propose an unsupervised method to discover long-tail categories in instance segmentation by learning instance embeddings of masked regions. By leveraging self-supervised losses for learning mask embeddings, the model trained on COCO dataset is able to identify novel and more fine-grained objects than common categories. The model achieves competitive quantitative results on LVIS compared to supervised and partially supervised methods.