ANOMALY DETECTION IN EM IMAGES - A ZERO-SHOT LEARNING APPROACH

Reconstructing dense neural wiring diagrams in the brain require 2D registration and 3D alignment ([1]) of thousands of serial sections imaged using a serial section electron microscopy (ssEM). Each serial section imaged comprise of thousands of high resolution images that include both the tissue, film, and resin regions. The stitching accuracy is challenged by the presence of anomalies (film and resin), which causes misalignments between the images that represent them. Detection of such anomalies is required to achieve a high quality stitching and alignment. Automated pixel based segmentation of such anomalies requires training machine learning models on dataset-specific ground truth, which is tedious to acquire. In this work, we propose a zero-shot learning approach to perform pixel based segmentation of resin and film regions on unknown datasets without ground truth. Our pipeline consists of a segmentation network based on U-Net architecture coupled with an unsupervised imageto-image translation method that uses Cycle GAN to translate the input image to match the style of images used for training the U-Net. Performance results on various EM datasets show that the segmentation class accuracy of resin and film improve significantly. By eliminating the need for ground-truth annotations, we facilitate the process of stitching and alignment by automation of detecting anomalies. Our zeroshot learning approach can potentially be utilized in scenarios where ground truth annotations are hard to acquire.

Automated summary

This paper proposes a zero-shot learning approach for pixel-based segmentation of resin and film regions in unknown datasets. By combining a U-Net segmentation network with an unsupervised image-to-image translation method, the segmentation accuracy is significantly improved. This approach automates the detection of anomalies, facilitating the reconstruction of dense neural wiring diagrams in the brain.