Multi-scale, multi-dimensional binocular endoscopic image depth estimation network

During invasive surgery, the use of deep learning techniques to acquire depth information from lesion sites in real-time is hindered by the lack of endoscopic environmental datasets. This work aims to develop a high-accuracy three-dimensional (3D) simulation model for generating image datasets and acquiring depth information in real-time. Here, we proposed an end-to-end multi-scale supervisory depth estimation network (MMDENet) model for the depth estimation of pairs of binocular images. The proposed MMDENet highlights a multi-scale feature extraction module incorporating contextual information to enhance the correspondence precision of poorly exposed regions. A multi-dimensional information-guidance refinement module is also proposed to refine the initial coarse disparity map. Statistical experimentation demonstrated a 3.14% reduction in endpoint error compared to state-of-the-art methods. With a processing time of approximately 30fps, satisfying the requirements of real-time operation applications. In order to validate the performance of the trained MMDENet in actual endoscopic images, we conduct both qualitative and quantitative analysis with 93.38% high precision, which holds great promise for applications in surgical navigation.

Automated summary

This study aims to develop a high-accuracy 3D simulation model for generating image datasets and acquiring real-time depth information in invasive surgery. An end-to-end multi-scale supervisory depth estimation network (MMDENet) is proposed for the depth estimation of binocular images. The proposed MMDENet utilizes a multi-scale feature extraction module and a multi-dimensional information-guidance refinement module, resulting in a 3.14% reduction in endpoint error compared to state-of-the-art methods.