Unsupervised 3-D Tensor Subspace Decomposition Network for Spatial-Temporal-Spectral Fusion of Hyperspectral and Multispectral Images

Due to sensor design limitations and the influence of weather factors, it is currently challenging to obtain remote sensing images with high temporal, spatial, and spectral resolution. Spatial-temporal-spectral fusion aims to integrate the temporal, spatial, and spectral information from multiple sources of remote sensing images to reconstruct a remote sensing image with high temporal, spatial, and spectral resolution. Existing methods typically require at least three types of data to achieve spatial-temporal-spectral fusion. However, acquiring remote sensing data observed at the same time poses significant difficulties. The major challenge lies in effectively utilizing hyperspectral images (HSIs) with low spatial and temporal resolution and multispectral images with high temporal and spatial resolution to reconstruct remote sensing images with high temporal, spatial, and spectral resolution. To address the aforementioned issues, we propose a novel unsupervised 3-D tensor subspace decomposition network. Our method incorporates the theory of 3-D tensor subspace decomposition, utilizing a 3-D hyperspectral/multispectral tensor subspace extraction network to predict the hyperspectral tensor subspace features with low spatial resolution missing at other times (to better understand, the missing moment is defined as time 2). Subsequently, the 3-D hyperspectral tensor subspace reconstruction network is employed along with the time 2 hyperspectral tensor subspace features with low spatial resolution and the time 2 MSI to reconstruct the time 2 HSI with high spatial resolution. In the experiment, we utilize three simulated datasets and two real datasets to evaluate the fusion performance of our proposed method. The results demonstrate that our method achieves high-quality fusion results, exhibits comparable performance, and has robustness and practicality.

Automated summary

This study proposes a method for reconstructing remote sensing images with high temporal, spatial, and spectral resolution by fusing the temporal, spatial, and spectral information from multiple sources of remote sensing images. By utilizing tensor subspace decomposition and reconstruction networks, it effectively utilizes low-resolution hyperspectral images and high-resolution multispectral images. Experimental results demonstrate that the method achieves high-quality fusion results, exhibits comparable performance, and has robustness and practicality.