Journal
IEEE TRANSACTIONS ON IMAGE PROCESSING
Volume 30, Issue -, Pages 1423-1438Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TIP.2020.3044214
Keywords
Spatial resolution; Image reconstruction; Tensors; Hyperspectral imaging; Residual neural networks; Principal component analysis; Optimization; Hyperspectral imagery; super-resolution; image fusion; deep learning; zero-mean normalization; cross-modality
Funding
- Hong Kong Research Grants Council [9048123 (CityU 21211518), 9042820 (CityU 11219019)]
- Macau Science and Technology Development Fund [077/2018/A2]
Ask authors/readers for more resources
This paper proposes a novel deep neural network framework, PZRes-Net, to efficiently address the problem of hyperspectral image super-resolution. PZRes-Net learns high-resolution and zero-centric residual images, explores coherence across all spectral bands, and outperforms state-of-the-art methods significantly in both quantitative metrics and visual quality in experiments.
This paper explores the problem of hyperspectral image (HSI) super-resolution that merges a low resolution HSI (LR-HSI) and a high resolution multispectral image (HR-MSI). The cross-modality distribution of the spatial and spectral information makes the problem challenging. Inspired by the classic wavelet decomposition-based image fusion, we propose a novel lightweight deep neural network-based framework, namely progressive zero-centric residual network (PZRes-Net), to address this problem efficiently and effectively. Specifically, PZRes-Net learns a high resolution and zero-centric residual image, which contains high-frequency spatial details of the scene across all spectral bands, from both inputs in a progressive fashion along the spectral dimension. And the resulting residual image is then superimposed onto the up-sampled LR-HSI in a mean-value invariant manner, leading to a coarse HR-HSI, which is further refined by exploring the coherence across all spectral bands simultaneously. To learn the residual image efficiently and effectively, we employ spectral-spatial separable convolution with dense connections. In addition, we propose zero-mean normalization implemented on the feature maps of each layer to realize the zero-mean characteristic of the residual image. Extensive experiments over both real and synthetic benchmark datasets demonstrate that our PZRes-Net outperforms state-of-the-art methods to a significant extent in terms of both 4 quantitative metrics and visual quality, e.g., our PZRes-Net improves the PSNR more than 3dB, while saving 2.3x parameters and consuming 15x less FLOPs. The code is publicly available at https://github.com/zbzhzhy/PZRes-Net
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available