Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion

Hyperspectral (HS) super-resolution, which aims at enhancing the spatial resolution of hyperspectral images (HSIs), has recently attracted considerable attention. A common way of HS super-resolution is to fuse the HSI with a higher spatial-resolution multispectral image (MSI). Various approaches have been proposed to solve this problem by establishing the degradation model of low spatial-resolution HSIs and MSIs based on matrix factorization methods, e.g., unmixing and sparse representation. However, this category of approaches cannot well construct the relationship between the high-spatial-resolution (HR) HSI and MSI. In fact, since the HSI and the MSI capture the same scene, these two image sources must have common factors. In this paper, a nonlocal tensor decomposition model for hyperspectral and multispectral image fusion (HSI-MSI fusion) is proposed. First, the nonlocal similar patch tensors of the HSI are constructed according to the MSI for the purpose of calculating the smooth order of all the patches for clustering. Then, the relationship between the HR HSI and the MSI is explored through coupled tensor canonical polyadic (CP) decomposition. The fundamental idea of the proposed model is that the factor matrices in the CP decomposition of the HR HSI's nonlocal tensor can be shared with the matrices factorized by the MSI's nonlocal tensor. Alternating direction method of multipliers is used to solve the proposed model. Through this method, the spatial structure of the MSI can be successfully transferred to the HSI. Experimental results on three synthetic data sets and one real data set suggest that the proposed method substantially outperforms the existing state-of-the-art HSI-MSI fusion methods.