Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum

Study objective Evaluate a universal proficiency metric for Robotic Surgery Virtual Reality (VR) simulation that will allow comparison of all users across any VR curriculum. Design Retrospective analysis of VR Simulation metrics. Setting Two training institutions. Patients or participants Residents, fellows and practicing surgeons. Interventions Analysis of the Mimic robotic Virtual Reality (VR)-Simulation database of over 600,000 sessions was utilized to calculate Mean scores for each exercise. Those Mean scores were then normalized to 100. Subject's scores were also averaged and normalized to 100. We called this Index score the MScore Proficiency Index (MPI(C)). Scores above 100 were better than average; Less than 100 were worse than average. Measurements and main results Seventeen thousand six hundred and forty eight sessions were analyzed (2017-2020) comparing 77 students (residents to practicing surgeons) working in 7 different curriculums. On average, each student spent 8 h and 24 min on simulation, attempted 26.5 different exercises, and became proficient in 20.6 exercises per user. The MPI(C) mean score for all participants in all curricula was an MPI(C) of 104.9 (SD: 15.5). Thirteen students were 1 standard deviation below the norm with an average MPI(C) of 80.15. This group averaged 9 h 27 min each on the simulator attempting 23.46 exercises but becoming proficient in only 10.38 (47%) of them in 224 sessions. Twelve students were 1 standard deviation above the norm with an average MPI(C) of 127.05. This group averaged 6 h 31 min each on the simulator attempting 29.08 exercises but becoming proficient in 27.5 (95%) of them in 196 sessions. Conclusion A universal skill-based performance index (MPI(C)) was calculated and found to be a reliable tool that could be used to identify relative proficiency among students in different robotic surgery VR Simulation curriculums. An individual user's proficiency can be utilized to identify a student's progress in a given curriculum. Future studies of MPI(C) will determine if machine learning can provide timely personalized feedback to the user.

Automated summary

A universal skill-based performance index (MPI(C)) was calculated and found to be a reliable tool for identifying relative proficiency among students in different robotic surgery VR simulation curriculums. Individual user proficiency can be used to track student progress in a given curriculum. Further studies will explore whether machine learning can provide personalized feedback for users.