Gradient structural similarity based gradient filtering for multi-modal image fusion

In the conventional structure tensor-based gradient domain image fusion methods, a structure tensor is exploited to calculate the fused gradient, from which the fused image can be derived using a variational model. However, in these conventional methods, because the direction of fused gradient at every position is determined by the inner product between the average of multiple source gradients and the biggest eigenvalue of structure tensor, its accuracy would be suffered by the canceling effect in calculating the average source gradient. To address such issue, we propose a novel local structural similarity metric to determine the dominant source gradient and correct the direction of fused gradient by the inner product between the biggest eigenvalue of structure tensor and the dominant source gradient. Moreover, in order to highlight salient features of the source images with the dominant source gradients, we propose a structural similarity based gradient filtering scheme which simultaneously performs filtering and fusion on both the source gradients and the corrected fused gradients to obtain the final fused gradients. Finally, the fused image can be reconstructed from the final fused gradients using a variational model like the conventional structure tensor-based fusion schemes. The comprehensive experiment results have revealed that our image fusion method can obtain better objective and subjective fusion performances compared to the state-of-the-art image fusion methods.