Fusion of Hyperspectral and Multispectral Images Accounting for Localized Inter-Image Changes

The high spectral resolution of hyperspectral images (HSIs) generally comes at the expense of low spatial resolution, which hinders the application of HSIs. Fusing an HSI and a multispectral image (MSI) from different sensors to get an image with the high spatial and spectral resolution is an economic and effective approach, but localized spatial and spectral changes between images acquired at different time instants can have negative impacts on the fusion results, which has rarely been considered in many fusion methods. In this article, we propose a novel group sparsity constrained fusion (GSFus) method to fuse hyperspectral and MSIs based on matrix factorization. Specifically, we imposed l(2,1) norm on the residual term of theMSI to account for the localized interimage changes occurring during the acquisition of the hyperspectral and MSIs. Furthermore, by exploiting the plug-and-play framework, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients. We refer to the proposed fusion method as GSFus method. We performed fusion experiments on two kinds of datasets, i.e., with and without obvious localized changes between the HSIs and MSIs, and a full resolution dataset. Extensive experiments in comparison with seven state-of-the-art fusion methods suggest that the proposed fusion method is more effective on fusing hyperspectral and MSIs than the competitors.

Automated summary

This article proposes a novel group sparsity constrained fusion method based on matrix factorization for fusing hyperspectral and multispectral images. By considering localized interimage changes and integrating an advanced denoiser, this method achieves better fusion results.