Correlation of Colorectal Surgical Skill With Patient Outcomes: A Cautionary Tale

BACKGROUND: Previous work has demonstrated a correlation between video ratings of surgical skill and clinical outcomes. Some have proposed the use of video review for technical skill assessment, credentialing, and quality improvement. OBJECTIVE: Before its adoption as a quality measure for colorectal surgeons, we must first determine whether video-based skill assessments can predict patient outcomes among specialty surgeons. DESIGN: Twenty-one surgeons submitted one representative video of a minimally invasive colectomy. Each video was edited to highlight key steps and then rated by 10 peer surgeons using a validated American Society of Colon and Rectal Surgeons assessment tool. Linking surgeons' ratings to a validated surgical outcomes registry, we assessed the relationship between skill and risk-adjusted complication rates. SETTINGS: The study was conducted with the Michigan Surgical Quality Collaborative, a statewide collaborative including 70 community, academic, and tertiary hospitals. PATIENTS: Patients included those who underwent minimally invasive colorectal resection performed by the participating surgeons. MAIN OUTCOME MEASURES: Main outcome measures included 30-day risk-adjusted postoperative complications. RESULTS: The average technical skill rating for each surgeon ranged from 2.6 to 4.6. Risk-adjusted complication rate per surgeon ranged from 9.9% to 33.1%. Patients of surgeons in the bottom quartile of overall skill ratings were older and more likely to have hypertension or to smoke; patients of surgeons in the top quartile were more likely to be immunosuppressed or have an ASA score of 3 or higher. After patient- and surgery-specific risk adjustment, there was no statistically significant difference in complication rates between the bottom and top quartile surgeons (17.5% vs 16.8%, respectively, p = 0.41). LIMITATIONS: Limitations included retrospective cohort design with short-term follow-up of sampled cases. Videos were edited to highlight key steps, and reviewers did not undergo training to establish norms. CONCLUSIONS: Our study demonstrates that video-based peer rating of minimally invasive colectomy was not correlated with postoperative complications among specialty surgeons. As such, the adoption of video review for use in credentialing should be approached with caution.

Automated summary

This study demonstrates that video-based peer rating of minimally invasive colectomy was not correlated with postoperative complications among specialty surgeons. Therefore, caution should be exercised when adopting video review for use in credentialing.