Effect of a deep learning-based automatic upper GI endoscopic reporting system: a randomized crossover study (with video)

Background and Aims: EGD is essential for GI disorders, and reports are pivotal to facilitating postprocedure diagnosis and treatment. Manual report generation lacks sufficient quality and is labor intensive. We reported and validated an artificial intelligence-based endoscopy automatic reporting system (AI-EARS). Methods: The AI-EARS was designed for automatic report generation, including real-time image capturing, diagnosis, and textual description. It was developed using multicenter datasets from 8 hospitals in China, including 252,111 images for training, 62,706 images, and 950 videos for testing. Twelve endoscopists and 44 endoscopy procedures were consecutively enrolled to evaluate the effect of the AI-EARS in a multireader, multicase, cross-over study. The precision and completeness of the reports were compared between endoscopists using the AI-EARS and conventional reporting systems. Results: In video validation, the AI-EARS achieved completeness of 98.59% and 99.69% for esophageal and gastric abnormality records, respectively, accuracies of 87.99% and 88.85% for esophageal and gastric lesion location records, and 73.14% and 85.24% for diagnosis. Compared with the conventional reporting systems, the AI-EARS achieved greater completeness (79.03% vs 51.86%, P < .001) and accuracy (64.47% vs 42.81%, P < .001) of the textual description and completeness of the photo-documents of landmarks (92.23% vs 73.69%, P < .001). The mean reporting time for an individual lesion was significantly reduced (80.13 +/- 16.12 seconds vs 46.47 +/- 11.68 seconds, P < .001) after the AI-EARS assistance. Conclusions: The AI-EARS showed its efficacy in improving the accuracy and completeness of EGD reports. It might facilitate the generation of complete endoscopy reports and postendoscopy patient management.

Automated summary

An artificial intelligence-based endoscopy automatic reporting system (AI-EARS) was developed and validated, showing its efficacy in improving the accuracy and completeness of EGD reports. It has the potential to facilitate the generation of complete endoscopy reports and postendoscopy patient management.