AI-enabled high-speed photoacoustic endomicroscopy through a multimode fibre

Photoacoustic (PA) endoscopy promises to be useful in a variety of clinical contexts including intravascular imaging, gastrointestinal tracts imaging and surgical guidance. Recent advancements of optical wavefront shaping allow the development of ultrathin endoscopy probes based on multimode optical fibres, which can provide higher spatial resolution than previously reported fibre bundle-based endoscopes. In this work, we developed a forward-viewing PA endomicroscopy imaging system and further improved its performance with a deep image prior (DIP) neural network. Laser was focused and scanned through a multimode fibre via wavefront shaping, in which a real-valued intensity transmission matrix approach was used for fibre characterisation, and a digital micromirror device (DMD) was used for light modulation. The excited ultrasound waves at the distal fibre tip were detected by an ultrasound transducer. High fidelity images of ex vivo mouse red blood cells were acquired. A DIP neural network was then used to improve the spatial resolution with unsupervised learning. Convolutional filters were used to learn features of low-level images as priors and reconstruct high-resolution images accordingly. The performance of the DIP approach was evaluated using a structural similarity index measure (SSIM) at a level of 0.85 with 25% effective pixels, which outperformed the bicubic method. The use of DIP allows reducing scanning positions by several times, and thus improves the speed of pixel-wise PA microscopy imaging. With further miniaturisation of the ultrasound detector, we anticipate that this system could be used for real-time guidance of minimally invasive surgeries by providing micro-structural, molecular, and functional information of tissue.

Automated summary

A forward-viewing photoacoustic (PA) endomicroscopy imaging system was developed and its performance was further improved using a deep image prior (DIP) neural network. Laser was focused and scanned through a multimode fibre via wavefront shaping, and ultrasound waves were detected by an ultrasound transducer. The DIP approach improved spatial resolution and achieved high fidelity images. The system could potentially be used for real-time guidance in minimally invasive surgeries.