TD-Net:unsupervised medical image registration network based on Transformer and CNN

Medical image registration is a fundamental task in computer-aided medical diagnosis. Recently, researchers have begun to use deep learning methods based on convolutional neural networks (CNN) for registration, and have made remarkable achievements in medical image registration. Although CNN based methods can provide rich local information on registration, their global modeling ability is weak to carry out the long distance information interaction and restrict the registration performance. The Transformer is originally used for sequence-to-sequence prediction. Now it also achieves great results in various visual tasks, due to its strong global modeling capability. Compared with CNN, Transformer can provide rich global information, in contrast, Transformer lacks of local information. To address Transformer lacks local information, we propose a hybrid network which is similar to U-Net to combine Transformer and CNN, to extract global and local information (at each level). Specifically, CNN is first used to obtain the feature maps of the image, and the Transformer is used as encoder to extract global information. Then the results obtained by Transformer encoding are connected to the upsampling process. The upsampling uses CNN to integrate local information and global information. Finally, the resolution is restored to the input image, and obtain the displacement field after several convolution layers. We evaluate our method on brain MRI scans. Experimental results demonstrate that our method improves the accuracy by 1% compared with the state-of-the-art approaches.

Automated summary

Medical image registration is a crucial task in computer-aided medical diagnosis, and recent research has focused on using deep learning methods to improve its accuracy. This paper introduces the use of Transformer, a powerful global modeling tool, to enhance medical image registration. By combining Transformer with CNN in a hybrid network, the proposed method is able to extract both local and global information, resulting in improved accuracy in brain MRI scans compared to state-of-the-art approaches.