Green Fluorescent Protein and Phase Contrast Image Fusion Via Detail Preserving Cross Network

In cell and molecular biology, the fusion of green fluorescent protein (GFP) and phase contrast (PC) images aims to generate a composite image, which can simultaneously display the functional information in the GFP image related to the molecular distribution of biological living cells and the structural information in the PC image such as nucleus and mitochondria. In this paper, we propose a detail preserving cross network (DPCN), which consists of a structural-guided functional feature extraction branch (SFFEB), a functional-guided structural feature extraction branch (FSFEB) and a detail preserving module (DPM), to address the GFP and PC image fusion issue. Unlike traditional parallel multi-branch architectures used for multiple inputs, the SFFEB and the FSFEB are interacted via a cross manner to fuse the functional information from the GFP image and the structural information from the PC image more adequately. Moreover, the DPM is composed of eight multi-scale convolutional blocks (MSCBs) associated with short, medium, and long skip connections to further extract the detail information from the source images. Experimental results on the popular Arabidopsis thaliana cell database demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both qualitative and quantitative evaluations. The proposed method is also extended to deal with the functional and structural image fusion issue in medical imaging, and the promising results obtained exhibit its good generalizability. The code of our method is available at https://github.com/yuliu316316/DPCN-Fusion.

Automated summary

The proposed method in this paper aims to address the fusion of functional and structural information in cell and molecular biology images through a detail preserving cross network. By utilizing a structural-guided functional feature extraction branch, a functional-guided structural feature extraction branch, and a detail preserving module with multi-scale convolutional blocks, the proposed method outperforms state-of-the-art methods in both qualitative and quantitative evaluations, demonstrating good generalizability in medical imaging fusion applications.