ACCURACY OF CHATGPT-4.0 IMAGE INPUT FDIAGNOSIS OF KERATITISOR

To assess the capabilities of the image input function of ChatGPT-4 (software update Sept 25, 2023) for diagnosing corneal ulcers/keratitis from images compared to a human expert.

Norfolk and Norwich University Hospital NHS Foundation Trust, UK. Three public ophthalmology resources were utilised (EyeRounds, EyeWiki, Wikipedia).

We selected anterior segment slit lamp images of keratitis and corneal ulcers ulcers available under creative commons licence, with or without fluorescein and subsequent use of blue filter A selection of images including marginal keratitis, adenoviral keratitis, herpetic keratitis, acanthomoebal keratitis, fungal keratitis, neutrotrophic ulcer, Mooren's ulcer, peripheral ulcerative keratitis and filamentary keratitis were included. The image associated with each case was directly inputted into the ChatGPT-4 image input function with the query “what is this?” followed by a repeat search with the prompt “what type of keratitis is this?”. Subsequent diagnosis was compared both with the original image descriptor and with a corneal specialist.

A total of 25 corneal slit lamp photographs were analysed. When ChatGPT-4 was asked what the photograph showed, preferred answers were keratitis in 7 cases (28%), opacity/haze in 5 cases (20%) and epithelial defect in 4 cases (16%). In 7 cases (28%) cases a different ocular pathology was recognised and a healthy eye/normal anatomy was recorded in 2 (8%) cases. In only 2 cases (8%), the correct subtype of keratitis was identified. Once prompted that the image was of keratitis, the most common reply became that it is challenging or unable to diagnose the type of keratitis (10 cases, 40%) combined with a descriptor of various types of keratitis that exist. However, the number of correct diagnoses improved to 4 cases (16%). In 3 cases (12%), ChatGPT-4 directly reported it could not make a diagnosis. With the exception of 2 cases, where ChatGPT-4 directly stated it could not help, the system acknowledged its limitations and advised to consult an ophthalmologist or eye care professional for a definitive diagnosis or if there was real life concern. In comparison, the corneal specialist diagnosed 22/25 (88%) of cases consistently with the image descriptor given. 

Keratitis/corneal ulcer image recognition by the image input function of the current iteration of ChatGPT-4.0 is presently guarded in its ability to accurately diagnose keratitis. AI tools based on multimodal large language models have potential in streamlining referrals and assisting in clinical recognition and with prompting can be a source of information for patients. However, as would be expected, in the first public iteration of the image input tool, it remains unreliable in its ability to classify anterior segment pathology. This study would be of interest of ophthalmologists implementing off-the-shelf artificial intelligence (AI) in the clinical practice.