Comprehensive Observations of Resident Evolution: A Novel Method for Assessing Procedure-Based Residency Training

Background: Assessment of surgical skills in the operating room remains a challenge. Increasing documentation requirements of the Accreditation Council for Graduate Medical Education are necessitating mechanisms to document trainee competence without hindering operative turnover. The authors created a comprehensive electronic resource to facilitate plastic surgery training program compliance with changes mandated by Next Accreditation System Milestones and the ACGME. Methods: In 2013, the authors implemented the Comprehensive Observations of Resident Evolution, or CORE, a Web-based tool to assess plastic surgery residents. It comprises a rapid electronic assessment of resident operating room performance completed after each surgery; a data dashboard displaying graphical summaries of resident progress by case, Milestone, or current procedural terminology code; and an electronic Milestones tracker (MileMarker), which enables ongoing trainee assessments. Results: From January through October of 2014, 24 residents completed nearly 1300 Operative Entrustability Assessments. Thirty-eight percent of residents reported more immediate feedback regarding operative performance. The assessment demonstrates construct validity, which distinguishes novice residents from experienced residents. Individual case data identify resident-specific operative strengths and weaknesses. Using assessment data, the first two Clinical Competency Committee reviews were 81 percent and 87 percent shorter than Milestones pilot test site reports (average, 11.5 and 8 minutes versus 60 minutes per resident, respectively). Conclusions: Comprehensive Observations of Resident Evolution is capable of capturing operative performance data on all operating room cases by primary current procedural terminology code. It increases immediate attending/trainee feedback and assessment transparency, enables trainee self-monitoring, and informs end-of-rotation reviews, programwide assessments, and tailoring of training to address specific needs. It is a valuable resource for tracking resident progress in real-time while maintaining compliance with evolving ACGME requirements.