Methods for real-time feature-guided image fusion of intrasurgical volumetric optical coherence tomography with digital microscopy

4D-microscope-integrated optical coherence tomography (4D-MIOCT) is an emer-gent multimodal imaging technology in which live volumetric OCT (4D-OCT) is implemented in tandem with standard stereo color microscopy. 4D-OCT provides ophthalmic surgeons with many useful visual cues not available in standard microscopy; however it is challenging for the surgeon to effectively integrate cues from simultaneous-but-separate imaging in real-time. In this work, we demonstrate progress towards solving this challenge via the fusion of data from each modality guided by segmented 3D features. In this way, a more readily interpretable visualization that combines and registers important cues from both modalities is presented to the surgeon.& COPY; 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

Automated summary

4D-microscope-integrated optical coherence tomography (4D-MIOCT) is a emerging multimodal imaging technology that combines real-time volumetric OCT (4D-OCT) with standard stereo color microscopy. 4D-OCT provides visual cues not available in standard microscopy, but it is challenging for surgeons to integrate cues from simultaneous-but-separate imaging in real-time. In this work, the fusion of data from each modality guided by segmented 3D features is demonstrated, presenting a more interpretable visualization that combines and registers important cues from both modalities to the surgeons.