PPI Edge Infused Spatial-Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

Multispectral filter array (MSFA) sensors provide a cost-effective and one-shot acquisition solution to obtain well-aligned multiband images, which are helpful for various optical and remote-sensing applications. However, the sparse spatial sampling rate and strong spectral cross correlation make MSFA image demosaicing a challenging problem. Therefore, it is essential to develop effective MSFA demosaicing solutions to reconstruct full-resolution and high-fidelity multispectral images from the raw mosaic image. In this article, we present a pseudo-panchromatic image (PPI) edge-infused spatial-spectral adaptive residual network (PPIE-SSARN) for MSFA image demosaicing. The proposed two-branch model deploys a residual subbranch to adaptively compensate for the spatial and spectral differences of reconstructed multispectral images and a PPI edge infusion subbranch to enrich the edge-related information. Moreover, we design an effective mosaic initial feature extraction module with a spatial- and spectral-adaptive weight-sharing strategy whose kernel weights can change adaptively with spatial locations and spectral bands to avoid artifacts and aliasing problems. Experimental results demonstrate the superiority of our proposed method, outperforming the state-of-the-art MSFA demosaicing approaches and achieving satisfying demosaicing results in terms of spatial accuracy and spectral fidelity. Our models and code will be publicly available.

Automated summary

This article proposes a pseudo-panchromatic image (PPI) edge-infused spatial-spectral adaptive residual network (PPIE-SSARN) for MSFA image demosaicing. The method compensates for the spatial and spectral differences of reconstructed multispectral images and enriches the edge-related information using a two-branch model. Experimental results demonstrate the superiority of the proposed method in spatial accuracy and spectral fidelity. The models and code will be publicly available.