Study highlights risks of AI misguidance in ophthalmology

Although AI diagnostic tools can assist ophthalmologists, their accuracy and reliability depend heavily on the clinician’s level of expertise, according to a study that found experienced corneal specialists are less influenced by incorrect AI guidance, whereas less experienced non-specialists are more likely to be misled.

The study evaluated 46 ophthalmologists—7 corneal specialists and 16 non-specialists—who reviewed 30 cases each of infectious and immunological keratitis. Participants assessed the presence of infection based on clinical images alone, followed by the AI’s classification. The AI’s accuracy was deliberately manipulated, with correct results in 70% of cases and incorrect ones in 30%.

Overall, the diagnostic accuracy of ophthalmologists remained consistent before and after using the AI system (75.2% vs 75.9%, P = 0.59). However, the data highlighted differences based on expertise. Corneal specialists maintained accuracy even when presented with incorrect AI guidance (60.3% vs 53.2%, P = 0.11). Non-specialists, however, experienced a significant decline in accuracy under similar conditions, dropping from 54.5% to 31.6% (P < 0.001).

Reference
Maehara H, Ueno Y, Yamaguchi T, et al. The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading. Sci Rep. 2025;15(1):1462. doi: 10.1038/s41598-025-85827-0. PMID: 39789113.