Journal
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
Volume 60, Issue -, Pages -Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TGRS.2021.3066425
Keywords
Pansharpening; Training; Spatial resolution; Multiresolution analysis; Image fusion; Deep learning; Optimization; Adaptive fusion; deep learning; image fusion; multiresolution analysis; pansharpening; remote sensing; variational models
Categories
Funding
- NSFC [61772003, 61702083, 61876203]
- Key Projects of Applied Basic Research in Sichuan Province [2020YJ0216]
- National Key Research and Development Program of China [2020YFA0714001]
Ask authors/readers for more resources
This article proposes a generic fusion framework for pansharpening which combines variational optimization with deep learning. The framework achieves higher accuracy and generalization ability compared to recent state-of-the-art methods.
Pansharpening refers to a spatio-spectral fusion of a lower spatial resolution multispectral (MS) image with a high spatial resolution panchromatic image, aiming at obtaining an image with a corresponding high resolution both in the domains. In this article, we propose a generic fusion framework that is able to weightedly combine variational optimization (VO) with deep learning (DL) for the task of pansharpening, where these crucial weights directly determining the relative contribution of DL to each pixel are estimated adaptively. This framework can benefit from both VO and DL approaches, e.g., the good modeling explanation and data generalization of a VO approach with the high accuracy of a DL technique thanks to massive data training. The proposed method can be divided into three parts: 1) for the VO modeling, a general details injection term inspired by the classical multiresolution analysis is proposed as a spatial fidelity term and a spectral fidelity employing the MS sensorx2019;s modulation transfer functions is also incorporated; 2) for the DL injection, a weighted regularization term is designed to introduce deep learning into the variational model; and 3) the final convex optimization problem is efficiently solved by the designed alternating direction method of multipliers. Extensive experiments both at reduced and full-resolution demonstrate that the proposed method outperforms recent state-of-the-art pansharpening methods, especially showing a higher accuracy and a significant generalization ability.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available