Developing an Objective Structured Assessment of Technical Skills for Laparoscopic Suturing and Intracorporeal Knot Tying

OBJECTIVE: To develop a teaching and assessment tool for laparoscopic suturing and intracorporeal knot tying. DESIGN AND SETTING: We designed an Objective Structured Assessment of Technical Skills (OSATS) tool that includes a procedure-specific checklist (PSC) and global rating scale (GRS) to assess laparoscopic suturing and intracorporeal knot-tying performance. Obstetrics and Gynecology residents at our institution were videotaped while performing a laparoscopic suturing and intracorporeal knot-tying task at a surgical simulation workshop. A total of 2 expert reviewers assessed resident performance using the OSATS tool during live performance and 1 month later using the videotaped recordings. OSATS scores were analyzed using the Wilcoxon rank-sum test. Data are presented as median scores (interquartile range [IQR]). Intrarater and interrater reliabilities were assessed using a Spearman correlation and are presented as an r correlation coefficient and p value. An r >= 0.8 was considered as a high correlation. After testing, we received feedback from residents and faculty to improve the OSATS tool as part of an iterative design process. PARTICIPANTS: In all, 14 of 21 residents (66.7%) completed the study, with 9 junior residents and 5 senior residents. RESULTS: Junior residents had a lower score on the PSC than senior residents did; however, this was not statistically significant (median = 6.0 [IQR: 4.0-10.0] and median = 13.0 [IQR: 10.0-13.0]; p = 0.09). There was excellent intrarater reliability with our OSATS tool (for PSC component, r = 0.88 for Rater 1 and 0.93 for Rater 2, both p < 0.0001; for GRS component, r = 0.85 for Rater 1 and 0.88 for Rater 2, both p < 0.0002). The PSC also has high interrater reliability during live evaluation (r = 0.92; p < 0.0001), and during the videotape scoring with r= 0.77 (p = 0.001). CONCLUSIONS: Our OSATS tool may be a useful assessment and teaching tool for laparoscopic suturing and intracorporeal knot-tying skills. Overall, good intrarater reliability was demonstrated, suggesting that this tool may be useful for longitudinal assessment of surgical skills. (C) 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.