Journal
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS
Volume 19, Issue -, Pages -Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LGRS.2022.3194257
Keywords
Transformers; Hyperspectral imaging; Task analysis; Computer architecture; Training; Superresolution; Feature extraction; Hyperspectral image super-resolution (HISR); image fusion; remote sensing; Transformer
Categories
Funding
- National Natural Science Foundation of China [12171072, 61702083, 61876203]
- Key Projects of Applied Basic Research in Sichuan Province [2020YJ0216]
- National Key Research and Development Program of China [2020YFA0714001]
Ask authors/readers for more resources
This paper proposes a Fusformer method based on the Transformer architecture to solve the hyperspectral image super-resolution problem. By utilizing self-attention mechanism, this method can globally explore the intrinsic relationship within features, achieving better performance than current state-of-the-art methods.
Hyperspectral image super-resolution (HISR) is to fuse a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI), aiming to obtain a high-resolution hyperspectral image (HR-HSI). Recently, various convolution neural network (CNN)-based techniques have been successfully applied to address the HISR problem. However, these methods generally only consider the relation of a local neighborhood by convolution kernels with a limited receptive field, thus ignoring the global relationship in a feature map. In this letter, we design a Transformer-based architecture (called Fusformer) for the HISR problem, which is the first attempt to apply the Transformer architecture to this task to the best of our knowledge. Because of the excellent ability of feature representations, especially by the self-attention (SA) in the Transformer, our approach can globally explore the intrinsic relationship within features. Considering the specific HISR problem, since the LR-HSI holds the primary spectral information, our method estimates the spatial residual between the upsampled low-resolution multispectral image (LR-MSI) and the desired HR-HSI, reducing the burden of training the whole data in a smaller mapping space. Various experiments show that our approach outperforms current state-of-the-art (SOTA) HISR methods. The code is available at https://github.com/J-FHu/Fusformer.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available