Contrastive Learning for Fine-Grained Ship Classification in Remote Sensing Images

Fine-grained image classification can be considered as a discriminative learning process where images of different subclasses are separated from each other while the same subclass images are clustered. Most existing methods perform synchronous discriminative learning in their approaches. Although achieving promising results in fine-grained visual classification (FGVC) in natural images, these methods may fail in fine-grained ship classification (FGSC) problem in remote sensing (RS) images due to the highly imbalanced fineness and imbalanced appearances of ships among subclasses. To tackle the issue, we propose an asynchronous contrastive learning-based method for effective FGSC. The proposed method, which we refer to as Push-and-Pull Network (P(2)Net), includes a push-out stage and a pull-in stage, where the first stage forces all the instances to be decorrelated and then the second one groups them into each subclass. A dual-branch network is designed to separate/decorrelate the images with each other, while an integration module is designed to aggregate the decorrelated images into their corresponding subclass together with a proxy-based module designed for acceleration. In this way, the correlation between subclasses can be decoupled, which in turn makes the final classification much easier. Our method can be trained end-to-end and requires no additional annotations other than category information. Extensive experiments are conducted on two large-scale FGSC datasets (FGSC-23 and FGSCR-42). Our method outperforms other state-of-the-art approaches. Ablation experiments also suggest the effectiveness of our design. Our code is available at https://github.com/WindVChen/Push-and-Pull-Network.

Automated summary

The article introduces an asynchronous contrastive learning-based method for effective fine-grained ship classification in remote sensing images. The method, called Push-and-Pull Network (P(2)Net), separates images using a dual-branch network and aggregates them into subclasses using an integration module. Experimental results demonstrate the effectiveness of the proposed method.