AWFLN: An Adaptive Weighted Feature Learning Network for Pansharpening

Deep learning (DL)-based pansharpening methods have shown great advantages in extracting spectral-spatial features from multispectral (MS) and panchromatic (PAN) images compared with traditional methods. However, most DL-based methods ignore the local inner connection between the source images and the high-resolution MS (HRMS) image, which cannot fully extract spectral-spatial information and attempt to improve the quality of fusion by increasing the complexity of the network. To solve these problems, a lightweight network based on adaptive weighted feature learning network (AWFLN) is proposed for pansharpening. Specifically, a novel detail extraction model is first built by exploring the local relationship between HRMS and source images, thereby improving the accuracy of details and the interpretability of the network. Guided by this model, we then design a residual multiple receptive-field structure to fully extract spectral-spatial features of source images. In this structure, an adaptive feature learning block based on spectral-spatial interleaving attention is proposed to adaptively learn the weights of features and improve the accuracy of the extracted details. Finally, the pansharpened result is obtained by a detail injection model in AWFLN. Numerous experiments are carried out to validate the effectiveness of the proposed method. Compared to traditional and state-of-the-art methods, AWFLN performs the best both subjectively and objectively, with high efficiency. The code is available at https://github.com/yotick/AWFLN.

Automated summary

DL-based pansharpening methods have advantages in extracting spectral-spatial features, but often ignore the local inner connection between source images and HRMS. To address this, a lightweight network based on AWFLN is proposed, which includes a detail extraction model and a residual multiple receptive-field structure to fully extract features. Experimental results show that AWFLN outperforms traditional and state-of-the-art methods in terms of subjective and objective evaluations.