Journal
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022)
Volume -, Issue -, Pages 17481-17490Publisher
IEEE COMPUTER SOC
DOI: 10.1109/CVPR52688.2022.01698
Keywords
-
Funding
- NSFC fund [61831014]
- Shenzhen Science and Technology Project [ZDYBH201900000002, CJGJZD20200617102601004]
- Westlake Foundation [2021B1501-2]
- Lochn Optics
Ask authors/readers for more resources
In this paper, we propose a novel framework MST for hyperspectral image (HSI) reconstruction, which combines spectral-wise similarity and mask-guided mechanism. Extensive experiments show that MST outperforms state-of-the-art methods on simulation and real datasets.
Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement in the coded aperture snapshot spectral imaging (CASSI) system. The HSI representations are highly similar and correlated across the spectral dimension. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. However, existing CNN-based methods show limitations in capturing spectral-wise similarity and long-range dependencies. Besides, the HSI information is modulated by a coded aperture (physical mask) in CASSI. Nonetheless, current algorithms have not fully explored the guidance effect of the mask for HSI restoration. In this papa; we propose a novel framework, Mask-guided Spectral-wise Transformer (MST), for HSI reconstruction. Specifically, we present a Spectral-wise Multi-head Self-Attention (S-MSA) that treats each spectral feature as a token and calculates self-attention along the spectral dimension. In addition, we customize a Mask-guided Mechanism (MM) that directs S-MSA to pay attention to spatial regions with high-fidelity spectral representations. Extensive experiments show that our MST significantly outperforms state-of-the-art (SOTA) methods on simulation and real HSI datasets while requiring dramatically cheaper computational and memory costs. https://github.com/caiyuanhao1998/MST/
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available