MCPT: Mixed Convolutional Parallel Transformer for Polarimetric SAR Image Classification

Vision transformers (ViT) have the characteristics of massive training data and complex model, which cannot be directly applied to polarimetric synthetic aperture radar (PolSAR) image classification tasks. Therefore, a mixed convolutional parallel transformer (MCPT) model based on ViT is proposed for fast PolSAR image classification. First of all, a mixed depthwise convolution tokenization is introduced. It replaces the learnable linear projection in the original ViT to obtain patch embeddings. The process of tokenization can reduce computational and parameter complexity and extract features of different receptive fields as input to the encoder. Furthermore, combining the idea of shallow networks with lower latency and easier optimization, a parallel encoder is implemented by pairing the same modules and recombining to form parallel blocks, which can decrease the network depth and computing power requirement. In addition, the original class embedding and position embedding are removed during tokenization, and a global average pooling layer is added after the encoder for category feature extraction. Finally, the experimental results on AIRSAR Flevoland and RADARSAT-2 San Francisco datasets show that the proposed method achieves a significant improvement in training and prediction speed. Meanwhile, the overall accuracy achieved was 97.9% and 96.77%, respectively.

Automated summary

A mixed convolutional parallel transformer (MCPT) model based on Vision transformers (ViT) is proposed for fast PolSAR image classification. The method introduces mixed depthwise convolution tokenization to reduce computational and parameter complexity and extract features of different receptive fields. It implements a parallel encoder by pairing the same modules and decreases the network depth and computing power requirement. Experimental results show significant improvement in training and prediction speed, with overall accuracy of 97.9% and 96.77% achieved on AIRSAR Flevoland and RADARSAT-2 San Francisco datasets, respectively.