SIMPLifying Urology Residency Operative Assessments: A Pilot Study in Urology Training

Purpose: The Society for Improving Medical Professional Learning (SIMPL) app is an innovative, convenient and validated smartphone-based tool to evaluate residents' operative performance. In this study, we describe the initial implementation of SIMPL in our program's pediatric urology rotationdthe first among urology residencies-and provide preliminary data on its adoption by residents and faculty. Materials and Methods: Residents and faculty in our pediatric urology division submitted SIMPL evaluations following surgical cases from August 2019 to July 2020. Evaluations consisted of ratings in 3 domains: resident autonomy, resident operative performance and patient-related case complexity. An online survey was also used to gauge attitudes towards SIMPL, describe patterns of use and solicit feedback on areas for improvement. Results: Eight residents and 6 faculty submitted 141 evaluations, with 76.6% of evaluated cases having both faculty and resident ratings. Verbal feedback was included in 94.2%. Faculty-resident agreement ranged from 68.6% to 75.2% (kappa=0.47 to 0.61). Faculty rated postgraduate year (PGY)-4 residents as more autonomous (p=0.040) and higher performing (p=0.028) than PGY-3 residents. All participants agreed that SIMPL was easy to use and compared favorably to existing avenues of feedback. Barriers to implementation included lack of reminders for evaluations and evaluation fatigue. Conclusions: The SIMPL application improved both frequency and quality of resident operative feedback. Among participants, SIMPL was preferred over the existing feedback system at our institution.

Automated summary

The study described the initial implementation of the SIMPL app in pediatric urology rotation, the first among urology residencies, and provided preliminary data on its adoption by residents and faculty. Results showed that the application improved both frequency and quality of resident operative feedback, with positive feedback from both faculty and residents, indicating it was preferred over existing feedback systems.