Do Metrics Matter? Time Versus Motion Tracking for Performance Assessment of Proficiency-Based Laparoscopic Skills Training

Introduction: The purpose of this study was to compare the added value of motion metrics in determining training completion during a proficiency-based simulator curriculum compared with traditional metrics (time). Methods: Novices (n = 16) practiced on a basic laparoscopic task of a hybrid simulator until expert-derived proficiency levels for time, path length, and smoothness were achieved on two consecutive attempts. The order by which proficiency in each metric was achieved was recorded and correlated to baseline characteristics. Motion metrics were considered valuable if their incorporation led to extension of training duration. Results: Compared with baseline participant performance improved at training completion according to all metrics (time 67 +/- 17 to 20 +/- 6 seconds; P < 0.001, pathlength 5326 +/- 1444 to 2339 +/- 545 cm; P < 0.001, and smoothness from 529 +/- 185 to 133 +/- 59; P < 0.001). Pathlength was the easiest metric to reach the proficiency level and time the most difficult. Four (33%) participants benefited from the motion metrics as their training was prolonged by an average of 50% compared with using time alone. Baseline characteristics did not correlate to the order of achievement of these metrics. Conclusions: Time may be superior to motion tracking metrics for performance assessment during proficiency-based simulator training. Nevertheless, in this study one third of trainees benefited from motion analysis metrics by having their training duration extended. Further study is needed to establish the value of motion metrics during simulator training and their impact on operating room performance improvement. (Sim Healthcare 4: 104-108, 2009)