4.7 Review

Computer vision in surgery: from potential to clinical value

Related references

Note: Only part of the references are listed.
Article Computer Science, Interdisciplinary Applications

Federated Cycling (FedCy): Semi-Supervised Federated Learning of Surgical Phases

Hasan Kassem et al.

Summary: Recent advancements in deep learning methods have brought safer surgical procedures closer through computer-assistance. Collaborative learning methods like Federated Learning (FL) have been proposed to allow training on remote datasets without explicitly sharing data. In this study, we propose FedCy, a federated semi-supervised learning method, to improve performance on surgical phase recognition by combining FL and self-supervised learning.

IEEE TRANSACTIONS ON MEDICAL IMAGING (2023)

Article Surgery

Artificial Intelligence for Surgical Safety Automatic Assessment of the Critical View of Safety in Laparoscopic Cholecystectomy Using Deep Learning

Pietro Mascagni et al.

Summary: This study developed a deep learning model to automatically segment hepatocystic anatomy and assess the criteria defining the critical view of safety in laparoscopic cholecystectomy. The results demonstrate that the model achieved reliable performance in segmentation and evaluation tasks.

ANNALS OF SURGERY (2022)

Article Surgery

Patient Perceptions of Audio and Video Recording in the Operating Room

Jean-Nicolas Gallant et al.

Summary: This study aimed to determine surgical patients' perceptions of continuous audio-video operating room recording (ORR). The findings showed that patients recognized the potential of recording to improve surgical quality, safety, and training. They also desired access to their own surgery recordings for future care and medical-legal evidence. However, there were mixed perceptions regarding the effect of ORR on OR behavior.

ANNALS OF SURGERY (2022)

Article Computer Science, Artificial Intelligence

Surgical data science-from concepts toward clinical translation

Lena Maier-Hein et al.

Summary: Surgical Data Science is a new research field aiming to improve interventional healthcare quality through data analysis, yet lacks translational success stories in surgery. An international workshop reviewed current practice, key achievements, infrastructure, data annotation and sharing, and data analytics standards and tools relevant to the field.

MEDICAL IMAGE ANALYSIS (2022)

Review Surgery

Video is better: why aren't we using it? A mixed-methods study of the barriers to routine procedural video recording and case review

Laura Mazer et al.

Summary: The majority of respondents believe that video can improve the quality of educational conferences, but also point out practical challenges in recording and editing video cases.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2022)

Review Surgery

Statistical models to preoperatively predict operative difficulty in laparoscopic cholecystectomy: A systematic review

Maria Vannucci et al.

Summary: This systematic review analyzes the performance and clinical value of statistical models in predicting the operative difficulty of laparoscopic cholecystectomy. The identified models have generally good performance and seem applicable, but an unambiguous definition of operative difficulty, validations, and clinical studies are needed.

SURGERY (2022)

Article Surgery

Artificial intelligence prediction of cholecystectomy operative course from automated identification of gallbladder inflammation

Thomas M. Ward et al.

Summary: The study found that an AI model can accurately identify the degree of gallbladder inflammation, which has an impact on the intra-operative course. The automated assessment system can be used for optimizing the workflow in the operating room and providing targeted feedback to surgeons and residents, accelerating the acquisition of operative skills.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2022)

Article Surgery

Multicentric validation of EndoDigest: a computer vision platform for video documentation of the critical view of safety in laparoscopic cholecystectomy

Pietro Mascagni et al.

Summary: This study validated the effectiveness of EndoDigest on a multicenter dataset of LC videos. The results showed that EndoDigest was able to accurately locate the time of the cystic duct division and efficiently document CVS. Despite the variability in surgical workflows across centers, EndoDigest demonstrated robustness and reliability.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2022)

Article Engineering, Biomedical

Video-based fully automatic assessment of open surgery suturing skills

Adam Goldbraikh et al.

Summary: The goal of this study was to develop a new reliable open surgery suturing simulation system using simple webcam video data for tools and hands localization, as well as identifying the interactions between them. The study developed a dual-task network for tool localization and tool-hand interaction detection. The results showed that the dual-task network performed similarly to two separate networks, with only a slight increase in computational load, and the motion metrics demonstrated significant differences between experts and novices.

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY (2022)

Article Computer Science, Artificial Intelligence

Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos

Chinedu Innocent Nwoye et al.

Summary: Action triplet recognition is a framework for surgical workflow analysis in endoscopic videos that aims to provide fine-grained and comprehensive information on surgical activities. To address the challenges of accurate identification and data association, a new model called RDV is introduced, which leverages attention at two different levels. Experimental results on a dataset show that the proposed RDV model significantly improves triplet prediction compared to state-of-the-art methods.

MEDICAL IMAGE ANALYSIS (2022)

Article Engineering, Biomedical

ClipAssistNet: bringing real-time safety feedback to operating rooms

Florian Aspart et al.

Summary: This article introduces a novel real-time feedback method to ensure safe visibility of the instrument during laparoscopic cholecystectomy, which can increase surgeons' attentiveness during the critical clipping of the cystic duct and artery.

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY (2022)

Review Surgery

The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review

Saba Balvardi et al.

Summary: This study aimed to systematically review the existing literature on the association between intraoperative technical performance measured using video-based assessment and patient outcomes. The results showed that better intraoperative performance was associated with fewer postoperative complications, reoperations, and readmissions. However, there were limited studies and mixed results when it came to assessing long-term outcomes.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2022)

Article Medical Informatics

Al recognition of patient race in medical imaging: a modelling study

Judy Wawira Gichoya et al.

Summary: In this study, we demonstrate that standard AI deep learning models can be trained to accurately predict a person's race from medical images across multiple imaging modalities. This detection is not attributed to proxies or imaging-related surrogate covariates for race, and the model's ability persists over all anatomical regions and frequency spectrums of the images. The findings highlight the risks associated with deploying AI models in medical imaging, as they can accurately predict race even from corrupted or manipulated images that clinical experts may struggle with.

LANCET DIGITAL HEALTH (2022)

Article Medical Informatics

The medical algorithmic audit

Xiaoxuan Liu et al.

Summary: The medical algorithmic audit framework is a tool for identifying potential errors in artificial intelligence systems, guiding auditors to consider clinical tasks, map components, and anticipate consequences. Testing algorithmic errors includes exploratory error analysis, subgroup testing, and adversarial testing.

LANCET DIGITAL HEALTH (2022)

Article Surgery

Artificial Intelligence for Intraoperative Guidance Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy

Amin Madani et al.

Summary: This study developed and evaluated artificial intelligence models that can identify safe and dangerous zones, as well as anatomical landmarks, during laparoscopic cholecystectomy (LC). The models showed promising results in accurately identifying these areas, indicating the potential for real-time guidance in surgery to minimize adverse events.

ANNALS OF SURGERY (2022)

Article Engineering, Biomedical

Surgical reporting for laparoscopic cholecystectomy based on phase annotation by a convolutional neural network (CNN) and the phenomenon of phase flickering: a proof of concept

M. Berlet et al.

Summary: This study proposes a workflow based on convolutional neural networks (CNN) to generate operative notes. By comparing the network's annotations with expert annotations and evaluating the usability of 'phase flickering', the results demonstrate the value of computer-aided documentation in phase recognition during standardized surgery.

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY (2022)

Article Surgery

Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy

Tatsushi Tokuyasu et al.

Summary: This study developed a system that outlines four landmarks in real time on endoscopic images during laparoscopic cholecystectomy (LC). The system was able to increase awareness of key landmarks among surgical teams and potentially reduce the incidence of bile duct injuries. Expert surgeons confirmed the validity of the system's indications, demonstrating its potential utility in clinical practice.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2021)

Review Surgery

The what? How? And Who? Of video based assessment

Carla M. Pugh et al.

Summary: There is significant variability in the development, implementation and overarching goals of video review for assessment of surgical performance. Despite advances in technology, there are barriers to successful implementation and longitudinal use of video-based assessment, including validity, scalability and discoverability.

AMERICAN JOURNAL OF SURGERY (2021)

Review Gastroenterology & Hepatology

Performance of artificial intelligence in colonoscopy for adenoma and polyp detection: a systematic review and meta-analysis

Cesare Hassan et al.

Summary: The meta-analysis found that incorporating artificial intelligence as an aid for detecting colorectal neoplasia significantly increases the detection rate of colorectal neoplasia, independently of the main adenoma characteristics.

GASTROINTESTINAL ENDOSCOPY (2021)

Article Surgery

Automated operative phase identification in peroral endoscopic myotomy

Thomas M. Ward et al.

Summary: The study successfully applied CV techniques to identify phases in endoscopic procedures, achieving an overall phase identification accuracy of 87.6% with the POEMNet model. The model performed well on a per-phase basis, showing the best accuracy for longer phases.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2021)

Review Surgery

Machine Learning for Surgical Phase Recognition A Systematic Review

Carly R. Garrow et al.

Summary: This study provides an overview of the use of ML models and data streams in automated surgical phase recognition. Through screening 2254 articles, the commonly used ML models and data sources were identified. The results demonstrate that ML can achieve high accuracy in surgical phase recognition.

ANNALS OF SURGERY (2021)

Article Surgery

A Computer Vision Platform to Automatically Locate Critical Events in Surgical Videos Documenting Safety in Laparoscopic Cholecystectomy

Pietro Mascagni et al.

Summary: The study aimed to develop a computer vision platform to automatically locate critical events in surgical videos and provide short video clips documenting the critical view of safety in laparoscopic cholecystectomy. The platform was successful in automatically locating the manually annotated time of the cystic duct division in full-length videos and extracting relevant short video clips for surgeon assessment, demonstrating its potential impact on safer surgical practices.

ANNALS OF SURGERY (2021)

Article Biochemistry & Molecular Biology

An algorithmic approach to reducing unexplained pain disparities in underserved populations

Emma Pierson et al.

Summary: A deep learning approach using knee X-rays can predict underserved populations' experienced pain and reduce racial and socioeconomic disparities. Algorithmic predictions suggest that much of underserved patients' pain stems from factors within the knee.

NATURE MEDICINE (2021)

Review Surgery

Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis

Roi Anteby et al.

Summary: The study evaluated the accuracy of deep learning networks in analyzing laparoscopic surgery videos, showing applications mainly in surgery or instrument recognition, phase recognition, and anatomy recognition. Deep learning holds potential in laparoscopic surgery, but is limited by methodologies.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2021)

Article Surgery

Intraoperative Time-Out to Promote the Implementation of the Critical View of Safety in Laparoscopic Cholecystectomy: A Video-Based Assessment of 343 Procedures

Pietro Mascagni et al.

Summary: The study showed that performing a short intraoperative time-out can improve the achievement rate of critical view of safety during laparoscopic cholecystectomy, leading to a reduction in bailout procedures and an increase in postoperative reporting rate.

JOURNAL OF THE AMERICAN COLLEGE OF SURGEONS (2021)

Article Computer Science, Artificial Intelligence

Semi-supervised learning with progressive unlabeled data excavation for label-efficient surgical workflow recognition

Xueying Shi et al.

Summary: In this paper, a novel two-stage Semi-Supervised Learning method SurgSSL is proposed for label-efficient Surgical workflow recognition, leveraging the inherent knowledge in unlabeled data for improved performance. The method first implicitly excavates motion knowledge from unlabeled data and then performs explicit excavation using pre-knowledge pseudo labeling, achieving superior results compared to existing methods.

MEDICAL IMAGE ANALYSIS (2021)

Article Multidisciplinary Sciences

Automation of surgical skill assessment using a three-stage machine learning algorithm

Joeel L. Lavanchy et al.

Summary: Surgical skills are crucial for clinical outcomes, and continuous training and feedback are necessary for improvement. This study introduces an automated method for assessing surgical skills using machine learning algorithms in laparoscopic cholecystectomy videos.

SCIENTIFIC REPORTS (2021)

Article Radiology, Nuclear Medicine & Medical Imaging

Artificial intelligence in radiology: 100 commercially available products and their scientific evidence

Kicky G. van Leeuwen et al.

Summary: Among the 100 CE-marked AI products in the field of radiology, 64 lack peer-reviewed evidence, with only 18 proving to have potential clinical impact.

EUROPEAN RADIOLOGY (2021)

Review Oncology

Surgical data science and artificial intelligence for surgical education

Thomas M. Ward et al.

Summary: Surgical data science aims to enhance the quality and value of interventional healthcare by capturing, organizing, analyzing, and modeling procedural data. With advancements in artificial intelligence, SDS can unlock augmented and automated coaching, feedback, assessment, and decision support in surgery.

JOURNAL OF SURGICAL ONCOLOGY (2021)

Article Surgery

SAGES consensus recommendations on an annotation framework for surgical video

Ozanan R. Meireles et al.

Summary: This study utilized a modified Delphi process to create a consensus survey on annotation of surgical video data, reaching consensus on recommendations within each domain through the collaboration of four working groups. The consensus recommendations presented a general framework for annotation, laying the foundation for standardization and enabling diverse datasets, performance benchmarks, and collaboration in surgical video analysis.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2021)

Article Surgery

Challenges in surgical video annotation

Thomas M. Ward et al.

Summary: Annotation of surgical video is crucial for establishing ground truth in surgical data science, but it faces challenges in annotating spatial, temporal, and clinical elements as well as selecting annotators. Reviewing these challenges can provide insights into opportunities for improvement and the potential next steps towards translating surgical data science efforts into clinical research and practice.

COMPUTER ASSISTED SURGERY (2021)

Review Computer Science, Information Systems

A Review on Deep Learning in Minimally Invasive Surgery

Irene Rivas-Blanco et al.

Summary: The research surveyed deep learning models applied to minimally invasive surgery, identifying different architectures, applications, results, and available datasets. The 85 publications were classified into four categories based on applications, with a technical description, comparison, and identification of future research directions provided.

IEEE ACCESS (2021)

Article Imaging Science & Photographic Technology

Hand Motion-Aware Surgical Tool Localization and Classification from an Egocentric Camera

Tomohiro Shimizu et al.

Summary: Detecting surgical tools in open surgery, where tools have similar shapes and are difficult to capture clearly, can be achieved by using hand movement data for classification. Combining localization, selection, and classification modules, the study successfully detected two different surgical tools.

JOURNAL OF IMAGING (2021)

Article Engineering, Biomedical

Patch-based classification of gallbladder wall vascularity from laparoscopic images using deep learning

Constantinos Loukas et al.

Summary: In this study, a deep learning approach was proposed for assessment of GB wall vascularity in LC images. The results showed high accuracy in patch and GB wall region classification. The study also investigated late fusion analysis techniques, providing a visual representation of the probability for vascularity classes across the GB wall region.

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY (2021)

Article Computer Science, Interdisciplinary Applications

Future-State Predicting LSTM for Early Surgery Type Recognition

Siddharth Kannan et al.

IEEE TRANSACTIONS ON MEDICAL IMAGING (2020)

Editorial Material Surgery

SAGES Video-Based Assessment (VBA) program: a vision for life-long learning for surgeons

Liane S. Feldman et al.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2020)

Article Multidisciplinary Sciences

Causality matters in medical imaging

Daniel C. Castro et al.

NATURE COMMUNICATIONS (2020)

Article Computer Science, Artificial Intelligence

On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities

Mauricio Reyes et al.

RADIOLOGY-ARTIFICIAL INTELLIGENCE (2020)

Review Surgery

Legal perspectives on black box recording devices in the operating environment

A. S. H. M. van Dalen et al.

BRITISH JOURNAL OF SURGERY (2019)

Article Computer Science, Interdisciplinary Applications

RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations

Andru Putra Twinanda et al.

IEEE TRANSACTIONS ON MEDICAL IMAGING (2019)

Article Engineering, Biomedical

Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy

Anita Rau et al.

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY (2019)

Article Engineering, Biomedical

Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos

Chinedu Innocent Nwoye et al.

INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY (2019)

Article Biochemistry & Molecular Biology

A guide to deep learning in healthcare

Andre Esteva et al.

NATURE MEDICINE (2019)

Article Computer Science, Theory & Methods

A survey on Image Data Augmentation for Deep Learning

Connor Shorten et al.

JOURNAL OF BIG DATA (2019)

Review Surgery

Outcome trends and safety measures after 30 years of laparoscopic cholecystectomy: a systematic review and pooled data analysis

Philip H. Pucher et al.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2018)

Article Computer Science, Artificial Intelligence

Improving Crowdsourced Label Quality Using Noise Correction

Jing Zhang et al.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2018)

Article Surgery

Understanding Costs of Care in the Operating Room

Christopher P. Childers et al.

JAMA SURGERY (2018)

Proceedings Paper Computer Science, Artificial Intelligence

Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks

Amy Jin et al.

2018 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2018) (2018)

Article Computer Science, Interdisciplinary Applications

EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos

Andru P. Twinanda et al.

IEEE TRANSACTIONS ON MEDICAL IMAGING (2017)

Meeting Abstract Medicine, General & Internal

Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes

Thomas G. Weiser et al.

LANCET (2015)

Editorial Material Surgery

Laparoscopic cholecystectomy: first, do no harm; second, take care of bile duct stones

George Berci et al.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2013)

Article Surgery

Effect of haptic feedback in laparoscopic surgery skill acquisition

M. Zhou et al.

SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES (2012)