Physics-based machine learning for subcellular segmentation in living cells

To train deep learning methods to segment very small subcellular structures, the training data have to be labelled by experts as the optical effects at such a small scale and the narrow depth of focus make it difficult to identify individual structures. Sekh et al. use a physics-based simulation approach to train neural networks to automatically segment subcellular structures despite the optical artefacts. Segmenting subcellular structures in living cells from fluorescence microscope images is a ground truth (GT)-deficient problem. The microscopes' three-dimensional blurring function, finite optical resolution due to light diffraction, finite pixel resolution and the complex morphological manifestations of the structures all contribute to GT-hardness. Unsupervised segmentation approaches are quite inaccurate. Therefore, manual segmentation relying on heuristics and experience remains the preferred approach. However, this process is tedious, given the countless structures present inside a single cell, and generating analytics across a large population of cells or performing advanced artificial intelligence tasks such as tracking are greatly limited. Here we bring modelling and deep learning to a nexus for solving this GT-hard problem, improving both the accuracy and speed of subcellular segmentation. We introduce a simulation-supervision approach empowered by physics-based GT, which presents two advantages. First, the physics-based GT resolves the GT-hardness. Second, computational modelling of all the relevant physical aspects assists the deep learning models in learning to compensate, to a great extent, for the limitations of physics and the instrument. We show extensive results on the segmentation of small vesicles and mitochondria in diverse and independent living- and fixed-cell datasets. We demonstrate the adaptability of the approach across diverse microscopes through transfer learning, and illustrate biologically relevant applications of automated analytics and motion analysis.

Automated summary

To solve the problem of segmenting very small subcellular structures, the study uses a physics-based simulation approach to train neural networks and introduces a simulation-supervision method supported by physics-based GT. This approach addresses the issue of lacking ground truth data and improves the accuracy and speed of subcellular segmentation.