Development and validation of an endoscopic submucosal dissection video assessment tool

Background Despite a need for assessment of endoscopic submucosal dissection (ESD) skills in order to track progress and determine competence, there is no structured measure of assessing competency in ESD performance. The present study aims to develop and examine validity evidence for an assessment tool to evaluate the recorded performance of ESD for gastric neoplasms. Methods The ESD video assessment tool (EVAT) was systematically developed by ESD experienced endoscopists. The EVAT consists of a 25-item global rating scale and 3-item checklist to assess competencies required to perform ESD. Five unedited videos were each evaluated by 2-blinded experienced ESD endoscopists to assess inter-rater reliability using intraclass correlation coefficients (ICC). Seventeen unedited videos in total were rated by 3 blinded experienced ESD endoscopists. Validity evidence for relationship to other variables was examined by comparing scores of inexperienced (fellows) and experienced endoscopists (attending staff), and by evaluating the relationship between the EVAT scores and ESD case experience. Internal consistency was evaluated using Cronbach's alpha. Results The inter-rater reliability for the total score was high at 0.87 (95% confidence interval 0.11 to 0.99). The total score [median, interquartile range (IQR)] was significantly different between the inexperienced (71, 63-77) and experienced group (95, 91-97) (P = 0.005). The total scores demonstrated high correlation with the number of ESD cases (Spearman's rho = 0.79, P < 0.01). The internal consistency was 0.97. Conclusions This study provides preliminary validity evidence for the assessment of video-recorded ESD performances for gastric neoplasms using EVAT.

Automated summary

This study developed and validated an assessment tool for evaluating video-recorded ESD performances for gastric neoplasms, demonstrating high inter-rater reliability and correlation with endoscopist experience, providing preliminary validity evidence for its use.