A prospective study in New Zealand has supported the potential application of Theia – an artificial intelligence (AI) diabetes retinopathy screening tool – as a clinical decision-making assistant when screening for diabetic retinopathy and maculopathy.
Theia was developed in New Zealand by Toku Eyes in partnership with the Auckland and Counties Manukau District Health Boards (DHB). It has been trained and tested locally, is clinic/clinician/camera agnostic, gender/age/ethnicity unbiased, and provides retinopathy and maculopathy grading to the New Zealand Ministry of Health requirements.
Theia successfully graded all images obtained during the study and identified every patient with ‘more than mild’ or ‘sight threatening’ disease
Published in Nature, the aim of this study was to establish the efficacy of Theia in a real-world prospective setting, regardless of the type of fundus camera being used, or the location of the screening centre.
To validate the screening tool, 900 patients attending an appointment as part of the New Zealand Diabetic Eye Screening Programme, were recruited from either an urban large eye hospital, or a semi-rural optometrist led screening provider. Their de-identified images were independently graded by three senior specialists, and final results were aggregated using the New Zealand grading scheme. Findings were then converted to referable/non-referable and healthy/mild/more than mild/sight threatening categories.
The authors reported that Theia successfully graded all images obtained during the study and identified every patient with ‘more than mild’ or ‘sight threatening’ disease.
When they compared the adjudicated images from the specialist grading team, ‘ground truth’, with the grading by the AI platform in detecting ‘sight threatening’ disease, they reported that “at the patient level Theia achieved 100% imageability, 100% [98.49–100.00%] sensitivity and [97.02–99.16%] specificity, and negative predictive value of 100%. The level of agreement between the clinicians and the aggregated results was (k value: 0.9881, 0.9557, and 0.9175), and the level of agreement between Theia and the aggregated labels was (k value: 0.9515).
They wrote, “As Theia has been tested on a variety of cameras, operating in a range of clinics (rural/urban, ophthalmologist-led\optometrist-led), we believe that it will be a suitable addition to a public diabetic screening program”. Additionally, they wrote, “with appropriate oversight and audit, these results indicate that Theia could be safely deployed within established diabetic screening programs to augment the expertise of the clinicians, increasing overall screening capacity while reducing costs per unit screen”.
Reference
Vaghefi, E., Yang, S., Xie, L. et al. A multi-centre prospective evaluation of THEIA™ to detect diabetic retinopathy (DR) and diabetic macular oedema (DMO) in the New Zealand screening program. Eye (2022). https://doi.org/10.1038/s41433-022-02217-w