Reliability and Initial Validation of the Ulcerative Colitis Endoscopic Index of Severity

BACKGROUND & AIMS: We studied the reliability of the previously described Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and validated it with an independent cohort of investigators. METHODS: We created a new library of 57 videos of flexible sigmoidoscopy and stratified them based on disease severity. Twenty-five investigators were each randomly assigned to assess 28 videos (which included 4 duplicates to assess intraobserver reliability). Investigators were blinded to clinical details except for 2 of 4 duplicated videos (to assess the impact of knowledge of symptoms on assessment). Three descriptors ( vascular pattern, bleeding, and erosions and ulcers) comprising the UCEIS were scored with a visual analogue scale (VAS) to assess overall severity. Intrainvestigator and interinvestigator agreement was characterized by k statistical analysis; reliability ratios were used to compare VAS and UCEIS scores. RESULTS: There was a high level of correlation between UCEIS scores and overall assessment of severity (correlation coefficient, 0.93). Internal consistency (Cronbach a analysis) was 0.86. Intrainvestigator and interinvestigator reliability ratios for UCEIS scores were 0.96 and 0.88, respectively. Intrainvestigator agreement in determination of the UCEIS score was good (k = 0.72), with individual descriptors ranging from a k of 0.47 (for bleeding) to 0.87 (for vascular pattern). Interinvestigator agreement in determination of UCEIS scores was moderate (k = 0.50), with descriptors ranging from a k of 0.48 (for bleeding) to 0.54 (for vascular pattern). Intrainvestigator variability in determining UCEIS scores did not change appreciably when a video was presented with clinical details. CONCLUSIONS: The UCEIS and its components show satisfactory intrainvestigator and interinvestigator reliability. Among investigators, the UCEIS accounted for a median of 86% of the variability in evaluation of overall severity on the VAS when assessing the endoscopic severity of UC and was unaffected by knowledge of clinical details.