期刊
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
卷 32, 期 3, 页码 1224-1235出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TCSVT.2021.3077058
关键词
Feature extraction; Decoding; Streaming media; Imaging; Sorting; Meteorology; Lighting; RGB-T data; salient object detection; cross-modality fusion; bilateral reversal fusion module; multilevel consistent fusion module
资金
- National Natural Science Foundation of China [61502429, 61972357]
- Zhejiang Provincial Natural Science Foundation of China [LY18F020012]
ECFFNet is a RGB-T feature fusion network that achieves accurate detection of salient objects by combining thermal images and RGB images. It outperforms state-of-the-art methods by implementing cross-modality fusion, bilateral reversal fusion, and multilevel consistent fusion.
Under ideal environmental conditions, RGB-based deep convolutional neural networks can achieve high performance for salient object detection (SOD). In scenes with cluttered backgrounds and many objects, depth maps have been combined with RGB images to better distinguish spatial positions and structures during SOD, achieving high accuracy. However, under low-light and uneven lighting conditions, RGB and depth information may be insufficient for detection. Thermal images are insensitive to lighting and weather conditions, being able to capture important objects even during nighttime. By combining thermal images and RGB images, we propose an effective and consistent feature fusion network (ECFFNet) for RGB-T SOD. In ECFFNet, an effective cross-modality fusion module fully fuses features of corresponding sizes from the RGB and thermal modalities. Then, a bilateral reversal fusion module performs bilateral fusion of foreground and background information, enabling the full extraction of salient object boundaries. Finally, a multilevel consistent fusion module combines features across different levels to obtain complementary information. Comprehensive experiments on three RGB-T SOD datasets show that the proposed ECFFNet outperforms 12 state-of-the-art methods under different evaluation indicators.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据