Journal
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
Volume 32, Issue 3, Pages 1224-1235Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TCSVT.2021.3077058
Keywords
Feature extraction; Decoding; Streaming media; Imaging; Sorting; Meteorology; Lighting; RGB-T data; salient object detection; cross-modality fusion; bilateral reversal fusion module; multilevel consistent fusion module
Categories
Funding
- National Natural Science Foundation of China [61502429, 61972357]
- Zhejiang Provincial Natural Science Foundation of China [LY18F020012]
Ask authors/readers for more resources
ECFFNet is a RGB-T feature fusion network that achieves accurate detection of salient objects by combining thermal images and RGB images. It outperforms state-of-the-art methods by implementing cross-modality fusion, bilateral reversal fusion, and multilevel consistent fusion.
Under ideal environmental conditions, RGB-based deep convolutional neural networks can achieve high performance for salient object detection (SOD). In scenes with cluttered backgrounds and many objects, depth maps have been combined with RGB images to better distinguish spatial positions and structures during SOD, achieving high accuracy. However, under low-light and uneven lighting conditions, RGB and depth information may be insufficient for detection. Thermal images are insensitive to lighting and weather conditions, being able to capture important objects even during nighttime. By combining thermal images and RGB images, we propose an effective and consistent feature fusion network (ECFFNet) for RGB-T SOD. In ECFFNet, an effective cross-modality fusion module fully fuses features of corresponding sizes from the RGB and thermal modalities. Then, a bilateral reversal fusion module performs bilateral fusion of foreground and background information, enabling the full extraction of salient object boundaries. Finally, a multilevel consistent fusion module combines features across different levels to obtain complementary information. Comprehensive experiments on three RGB-T SOD datasets show that the proposed ECFFNet outperforms 12 state-of-the-art methods under different evaluation indicators.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available