Transformer based multiple instance learning for WSI breast cancer classification

The computer-aided diagnosis method based on deep learning provides pathologists with preliminary di-agnostic opinions and improves their work efficiency. Inspired by the widespread use of transformers in computer vision, we try to explore their effectiveness and potential in classifying breast cancer tissues in WSIs, and propose a hybrid multiple instance learning method called HTransMIL. Specifically, its first stage is to select informative instances based on hierarchical Swin Transformer, which can capture global and local information of pathological images and is beneficial for obtaining accurate discriminative instances. The second stage aims to strengthen the correlation between selected instances via another transformer encoder consistently and produce powerful bag-level features by aggregating interactived instances for classification. Besides, visualization analysis is utilized to better understand the weakly supervised classification model for WSIs. The extensive evaluation results on a private and two public WSI breast cancer datasets demonstrate the effectiveness and competitiveness of HTransMIL. The code and models are publicly available at https: //github.com/Chengyang852/Transformer-for-WSI-classification.

Automated summary

Computer-aided diagnosis based on deep learning improves the efficiency of pathologists. This study explores the effectiveness of Transformers in classifying breast cancer tissues in WSIs and proposes a hybrid multiple instance learning method called HTransMIL. The method selects informative instances and strengthens their correlation to achieve accurate classification, while visualization analysis helps understand the weakly supervised classification model.