4.7 Article

STDFusionNet: An Infrared and Visible Image Fusion Network Based on Salient Target Detection

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TIM.2021.3075747

关键词

Deep learning; image fusion; infrared image; mask; salient target detection

资金

  1. National Natural Science Foundation of China [61773295, 62072223]
  2. Key Research and Development Program of Hubei Province [2020BAB113]
  3. Natural Science Foundation of Hubei Province [2019CFA037]

向作者/读者索取更多资源

The proposed STDFusionNet is a fusion network for infrared and visible images that preserves thermal targets and texture structures. By utilizing salient target detection and a specific loss function, it successfully extracts and reconstructs features to achieve high-quality fusion results. Through qualitative and quantitative experiments, the algorithm has shown superiority in terms of speed and quality improvement over existing methods.
In this article, we propose an infrared and visible image fusion network based on the salient target detection, termed STDFusionNet, which can preserve the thermal targets in infrared images and the texture structures in visible images. First, a salient target mask is dedicated to annotating regions of the infrared image that humans or machines pay more attention to, so as to provide spatial guidance for the integration of different information. Second, we combine this salient target mask to design a specific loss function to guide the extraction and reconstruction of features. Specifically, the feature extraction network can selectively extract salient target features from infrared images and background texture features from visible images, while the feature reconstruction network can effectively fuse these features and reconstruct the desired results. It is worth noting that the salient target mask is only required in the training phase, which enables the proposed STDFusionNet to be an end-to-end model. In other words, our STDFusionNet can fulfill salient target detection and key information fusion in an implicit manner. Extensive qualitative and quantitative experiments demonstrate the superiority of our fusion algorithm over the current state of the arts, where our algorithm is much faster and the fusion results look like high-quality visible images with clear highlighted infrared targets. Moreover, the experimental results on the public datasets reveal that our algorithm can improve the entropy (EN), mutual information (MI), visual information fidelity (VIF), and spatial frequency (SF) metrics with about 1.25%, 22.65%, 4.3%, and 0.89% gains, respectively. Our code is publicly available at https://github.com/jiayi-ma/STDFusionNet.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据