ACCURACY OF CATARACT DIAGNOSTICS WITH CHATGPT 4.0

Recently, ChatGPT 4.0 (software update 25th September 2023) has built-in image recognition capabilities, potentially diagnosing cataract from anterior segment photographs. In this project, we test the accuracy of ChatGPT 4.0 in identifying different types of cataracts to determine if it can be relied on for diagnostics and subsequent management of cataracts.

Norfolk and Norwich University Hospital NHS Foundation Trust, UK.

Anterior segment photographs of cataracts from publically available online resources including Eyewiki, Eyerounds and Wikipedia were uploaded onto ChatGPT 4.0, using the prompt "What type of cataract is this?". Cataracts used included: anterior polar, blue dot, brunescent, Christmas tree, congenital nuclear, cortical, intumescent, lamellar, Morganian, nuclear, posterior polar, posterior subcapsular, rosette, snowflake cataract, stellate, sunflower. The responses were analysed by the model and compared with the online description.

Of the 40 images, ChatGPT 4.0 correctly described 8 (20%) cataracts. With a tendency to favour ‘posterior subcapsular cataract’ as the incorrect answer for most images. Interestingly, when the image was in fact a “posterior subcapsular cataract”, the Artificial Intelligence (AI) model provided an incorrect diagnosis describing it in most cases as a “nuclear cataract”. The AI model did acknowledge its limitation in diagnosing cataracts, always stating that one should seek advice from an ‘eye healthcare professional’ for a definitive diagnosis.

While many studies have shown promising results in using AI and machine learning tools in cataract diagnosis, with very high rates of accuracy, in this study, ChatGPT 4.0 had much lower levels of accuracy. As we look to the future, AI models are likely to improve within medically orientated image recognition. These may be used as a referral screening mechanism to secondary care services. This study would be of interest to clinicians utilising ChatGPT 4.0 in their clinical workflow, and ophthalmologists should avoid using ChatGPT 4.0 to identify clinical signs with respect to cataract.