Modal complementary fusion network for RGB-T salient object detection

RGB-T salient object detection (SOD) combines thermal infrared and RGB images to overcome the light sensitivity of RGB images in low-light conditions. However, the quality of RGB-T images could be unreliable under complex imaging scenarios, and direct fusion of these low-quality images will lead to sub-optimal detection results. In this paper, we propose a novel Modal Complementary Fusion Network (MCFNet) to alleviate the contamination effect of low-quality images from both global and local perspectives. Specifically, we design a modal reweight module (MRM) to evaluate the global quality of images and adaptively reweight RGB-T features by explicitly modelling interdependencies between RGB and thermal images. Furthermore, we propose a spatial complementary fusion module (SCFM) to explore the complementary local regions between RGB-T images and selectively fuse multi-modal features. Finally, multi-scale features are fused to obtain the salient detection result. Experiments on three RGB-T benchmark datasets demonstrate that our MCFNet achieved outstanding performance compared with the latest state-of-the-art methods. We have also achieved competitive results in RGB-D SOD tasks, which proves the generalization of our method. The source code is released at https://github.com/dotaball/MCFNet.

Automated summary

This paper proposes a Modal Complementary Fusion Network (MCFNet) to address the issue of sub-optimal detection results caused by direct fusion of low-quality RGB-T images. It introduces a modal reweight module to evaluate the global quality of images and adaptively reweight RGB-T features, as well as a spatial complementary fusion module to selectively fuse multi-modal features. Experimental results demonstrate the outstanding performance of MCFNet compared to state-of-the-art methods.