Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology

PurposeTo assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians' perception about utilizing artificial intelligence (AI) in ophthalmology practice.MethodsSix cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done.ResultsUveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools.ConclusionsChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice.

Automated summary

ChatGPT demonstrated promising diagnostic capabilities in uveitis cases, and ophthalmologists showed enthusiasm for the integration of artificial intelligence into clinical practice.