Spectral Reconstruction From Satellite Multispectral Imagery Using Convolution and Transformer Joint Network

Spectral reconstruction (SR) based on satellite multispectral (MS) images can produce high spatial resolution hyperspectral (HS) images at a reasonable cost, significantly expanding the application of satellite-based HS remote sensing. As a challenging ill-posed problem, existing methods have difficulty making full use of local and global information of space and spectra to guide the reconstruction, resulting in limited accuracy in large-scale scenes with complex ground features and severe spectral mixing. In this article, we propose a novel convolution and transformer joint network (CTJN) to address the challenge of high-accuracy SR in complex scenes. The CTJN is cascaded with shallow feature extraction modules (SFEMs) and deep feature extraction modules (DFEMs), which can explore local spatial features and global spectral features. Besides, a high-frequency transformer block (HF-TB) is designed to highlight the detailed features of the images to prevent significant high-frequency information loss, which could improve the reconstruction results in regions with drastic feature changes. Moreover, a spatial-spectral recalibration block (SSRB) is proposed to perform explicit constraints on the reconstructed points by exploiting the correlation among neighboring pixels and adjacent spectra. Extensive experimental results on four HS-MS datasets and one MS dataset demonstrate that the proposed CTJN outperforms the state-of-the-art methods in large- and small-scale scenes.

Automated summary

In this article, a novel convolution and transformer joint network (CTJN) is proposed to address the challenge of high-accuracy spectral reconstruction (SR) in complex scenes. The CTJN utilizes shallow feature extraction modules (SFEMs) and deep feature extraction modules (DFEMs) to explore local spatial features and global spectral features. Additionally, a high-frequency transformer block (HF-TB) is designed to preserve detailed features and a spatial-spectral recalibration block (SSRB) is incorporated to enforce explicit constraints. Experimental results on multiple datasets demonstrate the superior performance of CTJN compared to state-of-the-art methods in both large- and small-scale scenes.