Interobserver Variation Study of the Rutgeerts Score to Assess Endoscopic Recurrence after Surgery for Crohn's Disease

Background: After resection surgery for Crohn's disease, recurrence of endoscopic lesions at the site of the anastomosis or in the neoterminal ileum is graded according to the Rutgeerts score (RS). The goal of this study was to test the interobserver variability for RS. Methods: Thirteen trained endoscopists evaluated the RS on 39 videotapes of patients who had undergone resection for Crohn's disease with an ileocolonic anastomosis 6 months earlier. Videotapes were randomly assigned to endoscopists through a balanced incomplete block design. Each videotape was scored independently by four endoscopists, and each endoscopist evaluated 12 videotapes, making a total of 156 videotape assessments. Reproducibility levels of the RS were assessed through unweighted kappa estimates among multiple raters. The proportion of inappropriate therapeutic initiation was estimated by randomly selecting one endoscopist for each videorecording, assuming that the majority of endoscopists correctly classified endoscopic recurrence. Results: The kappa estimates were 0.43 (95% confidence interval: 0.33-0.52) for the RS on a 5-grade scale, 0.47 (0.28-0.66) for RS < i2 vs. >= i2, and 0.64 (0.42-0.85) for RS <= i2 vs. > i2. The percentages of inappropriate therapeutic initiation were 12.8% (3.8-21.9) when initiation was triggered by a RS >= i2 and 8.3% (1.1-15.6) when initiation was triggered by a RS > i2 (p = 0.41). Conclusion: The reproducibility of the RS was moderate, especially when differentiating < i2 from >= i2, which may lead to incorrect therapeutic decisions in > 10% of patients.