Journal
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
Volume 27, Issue 7, Pages 3489-3500Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/JBHI.2023.3264819
Keywords
Image fusion; transformer; self-adaptive weight fusion; self-reconstruction
Ask authors/readers for more resources
Medical image fusion technology plays a crucial role in computer-aided diagnosis by extracting useful cross-modality cues and generating high-quality fused images. Existing methods mainly focus on fusion rules, leaving room for improvement in cross-modal information extraction. In this study, we propose a novel encoder-decoder architecture with three technical novelties, including attribute-based self-reconstruction tasks, a hybrid network combining CNN and transformer modules, and a self-adaptive weight fusion rule. Extensive experiments demonstrate satisfactory performance on medical image datasets and other multimodal datasets.
Medical image fusion technology is an essential component of computer-aided diagnosis, which aims to extract useful cross-modality cues from raw signals to generate high-quality fused images. Many advanced methods focus on designing fusion rules, but there is still room for improvement in cross-modal information extraction. To this end, we propose a novel encoder-decoder architecture with three technical novelties. First, we divide the medical images into two attributes, namely pixel intensity distribution attributes and texture attributes, and thus design two self-reconstruction tasks to mine as many specific features as possible. Second, we propose a hybrid network combining a CNN and a transformer module to model both long-range and short-range dependencies. Moreover, we construct a self-adaptive weight fusion rule that automatically measures salient features. Extensive experiments on a public medical image dataset and other multimodal datasets show that the proposed method achieves satisfactory performance.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available