Validation of an Objective Assessment Tool for Tonsillectomy in Otolaryngology Resident Training

Objective Create and validate an objective structured assessment of technical skills (OSATS) for otolaryngology residents learning how to perform a tonsillectomy. Study Design Multicenter prospective longitudinal validation study. Methods A multi-institutional study at six tertiary academic otolaryngology residency programs from July 2009 to May 2012. Using the modified Delphi technique, a panel of pediatric otolaryngologists created a tonsillectomy task-based checklist (TBC) for a tonsil OSATS using a 5-point Likert-type scale. Residents were assessed by pediatric otolaryngology staff at the time of surgery with the TBC and a global rating scale. Procedure time, patient age, number of previously performed tonsillectomies, and surgical technique were also collected. Results One hundred sixty-seven tonsil OSATS were completed for 38 residents, and competency was recorded for 99 (59.2%). Residents scored as competent had performed significantly more previous tonsillectomies than those deemed noncompetent, 44.4 +/- 35.6 and 13.5 +/- 11.6, respectively (P < .001). The mean overall score on the tonsil TBC was 4.0 +/- 0.8 and 2.6 +/- 1.0 for competent and noncompetent, respectively (P < .001). Higher number of tonsillectomies performed and mean tonsil TBC score significantly increased the likelihood of competency (P < .001). Each additional tonsillectomy performed increased the likelihood of achieving competency by 6.3% (P = .006, 95% confidence interval (CI): 1.330-1.110), and each 1.0 point increase in mean tonsil TBC score increased the likelihood of competency by a factor of 2.71 (P = .006, 95% CI:1.330-5.513). There is a 95% likelihood of competency at 48 tonsillectomies or a tonsil TBC score of 4.91. Conclusion The tonsil OSATS is a valid and feasible instrument to assess resident competency with tonsillectomy and provides timely objective feedback. Level of Evidence 4. Laryngoscope, 2020

Automated summary

The study demonstrates that the competency of residents in performing tonsillectomy is correlated with the number of previous procedures and their assessment scores, while the objective structured assessment tool can effectively provide feedback on their performance.