Pan-sharpening via a gradient-based deep network prior

Pan-sharpening is a domain-specific task of satellite imagery fusion. However, most traditional methods fuse the panchromatic image and the multispectral images in linear manners, which lead to severe spectral and spatial distortions. In the meanwhile, discriminative learning methods are limited in specialized satellites and tasks. In this paper, we make an attempt to integrate a deep prior with model-based optimization scheme for pan-sharpening. The proposed deep prior is based on a convolutional neural network which is composed of the proposed problem-specific recursive block and is trained in gradient domain. We plug the trained prior in place of the spatial preservation term in model-based optimization scheme, and address it with the alternating direction method of multipliers. Final experimental results demonstrate that the proposed model can overcome the restriction of linear model, and greatly reduce spectral and spatial distortions. Compared with several discriminative learning methods, our model tends to achieve promising generalization across different satellites.