Multi-scale unsupervised network for infrared and visible image fusion based on joint attention mechanism

Infrared and visible image fusion can synthesize complementary features of salient objects and texture details which are important for all-weather detection and other tasks. Nowadays, the deep learning based unsupervised fusion solutions are preferred and have obtained good results since the reference images for fusion tasks are not available. In the existing methods, some prominent features are missing in the fused images and the visual vitality needs to be improved. From this thought, attention mechanism is introduced to the fusion network. Especially, channel dimension and spatial dimension attention are jointed to supplement each other for feature extraction. Multiple attention branches emphasize on multi-scale features to complete the encoding. Skip connections are added to learn residual information. The multi-layer perceptual loss, the structure similarity loss and the content loss together construct the strong constraints for training. Comparative experiments with subjective and objective evaluations on 4 traditional and 9 deep learning based methods demonstrate the advantages of the proposed model.

Automated summary

Infrared and visible image fusion is important for all-weather detection. Deep learning based unsupervised fusion solutions with attention mechanism and multi-layer perceptual loss have shown advantages in improving the quality and visual effects of fused images.