Automatic and Objective Assessment of Motor Skills Performance in Flexible Bronchoscopy

Background: Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists. Objectives: The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion. Methods: Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS). Results: Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson's correlation, r = 0.43, p < 0.001) and a lower procedure time (Pearson's correlation, r = -0.90, p < 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, p < 0.001). Using the contrasting groups' method, a passing score of -0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures. Conclusions: MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance.

Automated summary

This study developed an objective and automatic assessment tool for motor skills performance in flexible bronchoscopy, called MoBSS, which measures lower arm and hand movements. MoBSS was found to correlate with diagnostic completeness and procedure time, and differentiate operator performance based on experience level, with potential to help trainees improve towards expert performance.