Convolution Structure Sparse Coding for Fusion of Panchromatic and Multispectral Images

Recently, sparse coding-based image fusion methods have been developed extensively. Although most of them can produce competitive fusion results, three issues need to be addressed: 1) these methods divide the image into overlapped patches and process them independently, which ignore the consistency of pixels in overlapped patches; 2) the partition strategy results in the loss of spatial structures for the entire image; and 3) the correlation in the bands of multispectral (MS) image is ignored. In this paper, we propose a novel image fusion method based on convolution structure sparse coding (CSSC) to deal with these issues. First, the proposed method combines convolution sparse coding with the degradation relationship of MS and panchromatic (PAN) images to establish a restoration model. Then, CSSC is elaborated to depict the correlation in the MS bands by introducing structural sparsity. Finally, feature maps over the constructed high-spatial-resolution (HR) and low-spatial-resolution (LR) filters are computed by alternative optimization to reconstruct the fused images. Besides, a joint HR/LR filter learning framework is also described in detail to ensure consistency and compatibility of HR/LR filters. Owing to the direct convolution on the entire image, the proposed CSSC fusion method avoids the partition of the image, which can efficiently exploit the global correlation and preserve the spatial structures in the image. The experimental results on QuickBird and Geoeye-1 satellite images show that the proposed method can produce better results by visual and numerical evaluation when compared with several well-known fusion methods.