期刊
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
卷 17, 期 6, 页码 3920-3928出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TII.2020.3011067
关键词
Endoscopes; Estimation; Three-dimensional displays; Cameras; Surgery; In vivo; Learning systems; Deep learning; depth estimation; ego-motion; endoscope; minimally invasive surgery (MIS); odometry; unsupervised learning
类别
资金
- National Natural Science Foundation of China [61903115]
- Collaborative Innovation Project of Colleges and Universities of Anhui Province [GXXT-2019-014, GXXT-2019-041]
- Fundamental Research Funds for the Central Universities [PA2019GDQT0021, PA2019GDZC0100, PA2020GDKC0020, TII-20-1301]
This study proposed a fully unsupervised learning method for depth and motion estimation using continuous monocular endoscopic video. By designing EndoMotionNet and EndoDepthNet models, considering timing information, and implementing a multimode fusion mechanism, the accuracy of depth and motion estimation can be significantly improved.
Three-dimensional display and virtual reality technology have been applied in minimally invasive surgery to provide doctors with a more immersive surgical experience. One of the most popular systems based on this technology is the Da Vinci surgical robot system. The key to build the in vivo 3-D virtual reality model with a monocular endoscope is an accurate estimation of depth and motion. In this article, a fully unsupervised learning method for depth and motion estimation using the continuous monocular endoscopic video is proposed. After the detection of highlighted regions, EndoMotionNet and EndoDepthNet are designed to estimate ego-motion and depth, respectively. The timing information between consecutive frames is considered with a long short-term memory layer by EndoMotionNet to enhance the accuracy of ego-motion estimation. The estimated depth value of the previous frame is used to estimate the depth of the next frame by EndoDepthNet with a multimode fusion mechanism. The custom loss function is defined to improve the robustness and accuracy of the proposed unsupervised-learning-based method. Experiments with the public datasets verify that the proposed unsupervised-learning-based continuous depth and motion estimation method can effectively improve the accuracy of depth and motion estimation, especially after processing the continuous frame.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据