ArtSeg-Artifact segmentation and removal in brightfield cell microscopy images without manual pixel-level annotations

Brightfield cell microscopy is a foundational tool in life sciences. The acquired images are prone to contain visual artifacts that hinder downstream analysis, and automatically removing them is therefore of great practical interest. Deep convolutional neural networks are state-of-the-art for image segmentation, but require pixel-level annotations, which are time-consuming to produce. Here, we propose ScoreCAM-U-Net, a pipeline to segment artifactual regions in brightfield images with limited user input. The model is trained using only image-level labels, so the process is faster by orders of magnitude compared to pixel-level annotation, but without substantially sacrificing the segmentation performance. We confirm that artifacts indeed exist with different shapes and sizes in three different brightfield microscopy image datasets, and distort downstream analyses such as nuclei segmentation, morphometry and fluorescence intensity quantification. We then demonstrate that our automated artifact removal ameliorates this problem. Such rapid cleaning of acquired images using the power of deep learning models is likely to become a standard step for all large scale microscopy experiments.

Automated summary

Brightfield cell microscopy is a fundamental tool in life sciences, but the acquired images often contain visual artifacts that can hinder downstream analysis. This study proposes a pipeline called ScoreCAM-U-Net, which can automatically segment and remove these artifacts with limited user input. The model is trained using only image-level labels, making the process significantly faster than traditional pixel-level annotation methods. The study demonstrates the existence of artifacts in different brightfield microscopy image datasets and shows that the automated artifact removal improves downstream analyses. This method has the potential to become a standard step in large scale microscopy experiments.