CSU-Net: A CNN-Transformer Parallel Network for Multimodal Brain Tumour Segmentation

Medical image segmentation techniques are vital to medical image processing and analysis. Considering the significant clinical applications of brain tumour image segmentation, it represents a focal point of medical image segmentation research. Most of the work in recent times has been centred on Convolutional Neural Networks (CNN) and Transformers. However, CNN has some deficiencies in modelling long-distance information transfer and contextual processing information, while Transformer is relatively weak in acquiring local information. To overcome the above defects, we propose a novel segmentation network with an encoder-decoder architecture, namely CSU-Net. The encoder consists of two parallel feature extraction branches based on CNN and Transformer, respectively, in which the features of the same size are fused. The decoder has a dual Swin Transformer decoder block with two learnable parameters for feature upsampling. The features from multiple resolutions in the encoder and decoder are merged via skip connections. On the BraTS 2020, our model achieves 0.8927, 0.8857, and 0.8188 for the Whole Tumour (WT), Tumour Core (TC), and Enhancing Tumour (ET), respectively, in terms of Dice scores.

Automated summary

Medical image segmentation techniques play a crucial role in medical image processing and analysis. This study proposes a novel segmentation network called CSU-Net, which merges CNN and Transformer for feature extraction and utilizes dual Swin Transformer decoder blocks for feature upsampling, achieving accurate segmentation of brain tumor images.