Transformers in medical image segmentation: A review

Background and Objectives: Transformer is a model relying entirely on self-attention which has a wide range of applications in the field of natural language processing. Researchers are beginning to focus on the transformer in medical images due to the past few years having seen the rapid development of transformer in many vision fields such as vision transformer (ViT) and Swin transformer. In the last year, moreover, many scholars have applied transformer to medical image segmentation and have achieved good segmentation results. Transformer-based medical image segmentation has become one of the hot spots in this field. The purpose of this work is to categorize and review the segmentation methods of Unet-based transformer and other model based transformer in medical images.Methods: This paper summarizes the transformer-based segmentation models in the abdominal organs, heart, brain, and lung based on the relevant studies in the last two years. We described and analyzed the model structure including the position of the transformer in the model, the changes made by scholars to transformer and the combination with the model. In this work, the segmentation performance results based on Dice evaluation metrics are compared.Results: Through the help of 93 references, we find that researchers prefer to use Unet-based transformer models than others and place the transformer structure in the encoder. These new models improve the segmentation performance compared with U-Net and other segmentation models. However, there are not many related studies on lungs, which points to a new way for future research.Conclusions: We found that the combination of U-Net and transformer is more suitable for segmentation. In future research on medical image segmentation, researchers can use a suitable transformer-based segmentation method or modify the transformer structure according to the segmentation requirements. We hope that this work will be helpful for improvements of the transformer to solve clinical problems in medicine.

Automated summary

This study summarizes the segmentation methods based on Transformer in medical images of abdominal organs, heart, brain, and lungs in the past two years. The findings show that Unet-based Transformer models are preferred by researchers, and placing the Transformer structure in the encoder improves segmentation performance. However, there is a lack of related studies on lungs, indicating a new direction for future research.