An objective approach to evaluate novice robotic surgeons using a combination of kinematics and stepwise cumulative sum (CUSUM) analyses

Background Current evaluation methods for robotic-assisted surgery (ARCS or GEARS) are limited to 5-point Likert scales which are inherently time-consuming and require a degree of subjective scoring. In this study, we demonstrate a method to break down complex robotic surgical procedures using a combination of an objective cumulative sum (CUSUM) analysis and kinematics data obtained from the da Vinci (R) Surgical System to evaluate the performance of novice robotic surgeons. Methods Two HPB fellows performed 40 robotic-assisted hepaticojejunostomy reconstructions to model a portion of a Whipple procedure. Kinematics data from the da Vinci (R) system was recorded using the dV Logger (R) while CUSUM analyses were performed for each procedural step. Each kinematic variable was modeled using machine learning to reflect the fellows' learning curves for each task. Statistically significant kinematics variables were then combined into a single formula to create the operative robotic index (ORI). Results The inflection points of our overall CUSUM analysis showed improvement in technical performance beginning at trial 16. The derived ORI model showed a strong fit to our observed kinematics data (R-2 = 0.796) with an ability to distinguish between novice and intermediate robotic performance with 89.3% overall accuracy. Conclusions In this study, we demonstrate a novel approach to objectively break down novice performance on the da Vinci (R) Surgical System. We identified kinematics variables associated with improved overall technical performance to create an objective ORI. This approach to robotic operative evaluation demonstrates a valuable method to break down complex surgical procedures in an objective, stepwise fashion. Continued research into objective methods of evaluation for robotic surgery will be invaluable for future training and clinical implementation of the robotic platform.

Automated summary

This study demonstrated a novel approach to objectively evaluate novice performance on the da Vinci Surgical System by combining kinematics data and cumulative analysis to create an operative robotic index (ORI). The ORI model showed a strong fit to observed kinematics data and had high accuracy in distinguishing between novice and intermediate robotic performance. This objective method of evaluation provides valuable insight into complex surgical procedures for training and clinical implementation of robotic surgery.