Optical flow driven interpolation for isotropic FIB-SEM reconstructions

Background and Objective: Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) allows threedimensional ultrastructural analysis of cells and tissues at the nanoscale. The technique iteratively removes a section of the sample with a FIB and takes an SEM image from the exposed surface. The section thickness is usually higher than the image pixel size to reduce acquisition time, thus resulting in anisotropic resolution. In this work, we explore novel interpolation methods along the sectioning direction to produce isotropic resolution and facilitate proper interpretation of the FIB-SEM 3D volumes.Methods: Classical interpolation methods are usually applied in this context under the assumption that the changes through successive images are relatively smooth. However, the actual 3D arrangement of the structures in the sample may cause significant changes in the biological features between consecutive images of the FIB-SEM stacks. We have developed a novel interpolation strategy that accounts for this variation by using the Optical Flow (OF) to estimate it. As an intermediate stage, OF-compensated images are produced by aligning the spatial regions of the biological structures. Interpolated images are then generated from these OF-compensated images. The final isotropic stack is assembled by interleaving the interpolated images with the original images of the anisotropic stack.Results: OF-driven and classical interpolation methods were compared using an objective assessment based on Pearson Correlation Coefficient (PCC) and a qualitative evaluation based on visual results, using public datasets and representative anisotropy conditions. The objective assessment demonstrated that the OF-driven interpolation always yields higher PCC values, with interpolated images closer to the ground truth. The qualitative evaluation corroborated those results and confirmed that classical interpolation may blur areas with substantial changes between consecutive images whereas OF-driven interpolation provides sharpness.Conclusions: We have developed an OF-driven interpolation approach to generating FIB-SEM stacks with isotropic resolution from experimental anisotropic data. It adapts to the rapid variation of the biological structures observed through the images of the FIB-SEM stack. Our approach outperforms classical interpolation and manages to produce sharp interpolated views in cases where there are significant changes between consecutive experimental images. (c) 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )

Automated summary

In this study, an Optical Flow (OF)-driven interpolation strategy was developed to address the significant changes between consecutive images in FIB-SEM stacks. The approach compensated for variations by aligning the spatial regions of the biological structures using OF, resulting in sharp interpolated images. The OF-driven interpolation outperformed classical interpolation methods and produced isotropic resolution in anisotropic data.