4.7 Article

Multimodal Triplet Attention Network for Brain Disease Diagnosis

期刊

IEEE TRANSACTIONS ON MEDICAL IMAGING
卷 41, 期 12, 页码 3884-3894

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TMI.2022.3199032

关键词

Brain disease diagnosis; multi-modal data fusion; triplet siamese network; attention mechanism; high-order information

资金

  1. National Natural Science Foundation of China [62076129, 61902183, 62136004, 61501230, 81871345, 81790653]
  2. National Key Research and Development Program of China [2018YFC2001600, 2018YFC2001602, 2018YFA0701703]

向作者/读者索取更多资源

Multi-modal imaging data fusion is important in medical data analysis for more accurate analysis with complementary information. This paper proposes a novel method for epilepsy diagnosis by fusing data from functional MRI and diffusion tensor imaging, capturing complementary information and discriminative features. Experimental results show that the proposed method is significantly superior to other approaches.
Multi-modal imaging data fusion has attracted much attention in medical data analysis because it can provide complementary information for more accurate analysis. Integrating functional and structural multi-modal imaging data has been increasingly used in the diagnosis of brain diseases, such as epilepsy. Most of the existing methods focus on the feature space fusion of different modalities but ignore the valuable high-order relationships among samples and the discriminative fused features for classification. In this paper, we propose a novel framework by fusing data from two modalities of functional MRI (fMRI) and diffusion tensor imaging (DTI) for epilepsy diagnosis, which effectively captures the complementary information and discriminative features from different modalities by high-order feature extraction with the attention mechanism. Specifically, we propose a triple network to explore the discriminative information from the high-order representation feature space learned from multi-modal data. Meanwhile, self-attention is introduced to adaptively estimate the degree of importance between brain regions, and the cross-attention mechanism is utilized to extract complementary information from fMRI and DTI. Finally, we use the triple loss function to adjust the distance between samples in the common representation space. We evaluate the proposed method on the epilepsy dataset collected from Jinling Hospital, and the experiment results demonstrate that our method is significantly superior to several state-of-the-art diagnosis approaches.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据