Deep learning 2D and 3D optical sectioning microscopy using cross-modality Pix2Pix cGAN image translation

Structured illumination microscopy (SIM) reconstructs optically-sectioned images of a sample from multiple spatially-patterned wide-field images, but the traditional single non patterned wide-field images are more inexpensively obtained since they do not require generation of specialized illumination patterns. In this work, we translated wide-field fluorescence microscopy images to optically-sectioned SIM images by a Pix2Pix conditional generative adversarial network (cGAN). Our model shows the capability of both 2D cross-modality image translation from wide-field images to optical sections, and further demonstrates potential to recover 3D optically-sectioned volumes from wide-field image stacks. The utility of the model was tested on a variety of samples including fluorescent beads and fresh human tissue samples. (c) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Automated summary

This study utilizes a Pix2Pix cGAN model to translate wide-field fluorescence microscopy images into optical section SIM images, demonstrating the potential for 2D to 3D image conversion and testing on fluorescent beads and human tissue samples.