A clinical artificial intelligence (AI) research group in the United Kingdom has created RETFound – the first foundation model of ophthalmology and one of the first in healthcare.
Pearse Keane, Professor of Artificial Medical Intelligence, Institute of Ophthalmology, Faculty of Brain Sciences University College London (UK), spoke about the development of the AI application at a breakfast hosted by Roche at the Royal Australian and New Zealand College of Ophthalmologists (RANZCO) Scientific Congress in Perth in October.
He and his colleagues have built the cloud-based infrastructure using just under two million historic anonymised eye scans, to identify signs of eye disease and recommend how patients should be referred for care. Available as open source, it is hoped that RETFound can become a cornerstone for global efforts to use AI to prevent blindness.
MEETING A GROWING NEED
While acknowledging that the increasing use of optical coherence tomography (OCT) in routine primary eye care is invaluable, Prof Keane said it has also increased the number of referrals to hospitals. This, combined with an aging population, has meant that ophthalmology has overtaken orthopaedics as the number one busiest of all the medical specialties in the United Kingdom in terms of outpatient appointments.
Prof Keane highlighted that AI has real potential “to bring expertise out of specialised centres, such as Moorfields Eye Hospital and into the community, to general ophthalmologists and optometrists, and upskill them in the way that they can deal with patients”.
He hopes it will revolutionise the way professionals carry out eye tests, allowing them to spot conditions earlier and prioritise patients with the most serious eye diseases before irreversible damage sets in.
A FOUNDATIONAL MODEL
With the evolution of technology and a massive 22 million scans available to work with (Moorfields Eye Hospital processes around 1,000 scans every day), Dr Keane said what is most exciting about building an AI application today is that instead of requiring human experts to label data and train the model (supervised learning), it is now possible for self-supervised learning without labelled data.
Additionally, the amount of data that can be processed and analysed is almost beyond comprehension. As he explained, this is being enabled by a shift from traditional ‘convolutional neural network architecture’ towards ‘transformer architecture’.
“So you’re seeing transformer models with up to a trillion parameters, a trillion mathematical calculations,” he said.
With this unlabelled approach, the group trained the application using just under two million retinal photographs or OCT scans. They covered 80% of the images randomly, and asked the model to try to predict what had been covered. They then showed that the model could be “fine-tuned for more than 10 different downstream clinical tasks with tiny amounts of labelled data. So that could be for diabetic retinopathy screening, for glaucoma, for predicting progression of AMD (age-related macular degeneration) or doing a range of what we call omics tasks”.
Having created the model with almost two million images, in the coming year or two the hope is to scale up to 20 million images.
Prof Keane said, “When we scale it up to 20 million images and beyond, we start to see emerging capabilities that they’ve seen in other fields. So, the models start to have power that you sort of hadn’t originally anticipated. We’re going to make it multimodal… so it can be trained on widefield images, near infrared, fundus auto fluorescence; and with a single backbone, train the model in such a way that it can easily be used on new imaging modalities, new devices, etcetera… We’re also developing a 3D version”.
LOGISTICS, ETHICS AND THE FUTURE
The Roche breakfast meeting progressed with a discussion on the logistics and ethics of data collection, sharing and security. The momentum of multimodal technologies and use of AI in medicine requires health care providers to carefully consider how they will integrate this technology into their clinical practice in a way that maximises outcomes for patients while protecting confidentiality and achieving efficiencies.
Sydney ophthalmologist and Big Data expert Dr Ashley Kras explained how the future and real opportunity with AI lies in combining multimodal clinical systems in real-time. A modern, next generation standard – FHIR (Fast Healthcare Interoperability Resource), which has a native DICOM interface for imaging – enables this integration and exponentially propels AIdriven precision medicine.
However, pragmatically, this relies on the clinical community and industry to conform with and contribute towards an ongoing need for standards development.
“Advances in healthcare technology have created exponential growth in the amount of data available for patient care from an increasing number of sources and our ability to access and use the data in real time hasn’t kept up,” Dr Kras said.
“When data is stored in non-compatible formats without standardisation within and between clinics – think primary care to optometry to ophthalmology, from imaging system to EMR (electronic medical record) – how is it possible to accurately and efficiently track a patient’s progress over time or apply data analytics to improve clinical decision support? Unstructured data presents a significant hurdle for healthcare professionals attempting to leverage data analytics to improve patient outcomes and healthcare operations.”
To overcome this he said, “we need standards development that defines the clinical lexicon to be used for data exchange and the API-based protocols that make this happen.”
With this in mind, Dr Kras has spearheaded the creation of an eye-care specific FHIR implementation guide ‘Eyes on FHIR’. His guide has garnered broad stakeholder endorsement and international support from leading academics, providers and industry leaders.1
“FHIR is closing the gap between the explosion of healthcare data and our ability to make that data accessible, computable and usable to improve outcomes.
“Its purpose is to enable seamless transfer of information in healthcare, just like you do when you automatically transfer money from one bank and branch to another through an app, or book travel with multiple providers through Webjet. “FHIR provides a set of rules and specifications for exchanging electronic health care data across a wide range of settings, which has already become mandatory in order to become a certified IT health vendor in the United States,” Dr Kras explained.
REDUCE ERRORS, DRIVE EFFICIENCY
He said standards-based interoperability “empowers clinicians, researchers and all healthcare stakeholders, including patients, payers, government and life science”.
“Combined with AI, FHIR has the potential to reduce medical errors, generate enormous efficiency gains in our healthcare system, and drive innovation in ways that will improve everything from the way a patient is diagnosed to how they are treated.
“The majority of AI development in ophthalmology has been almost exclusively image driven. But if you add even a basic parameter such as age, the baseline OCT between a 20-year-old person of a certain demographic versus an eight-year-old of a different demographic will inherently be different. Amplified through all of the clinical parameters that can be combined with imaging, this technology leads us to more personalised utilisation of AI,” he said.
“We’ve always been able to use inputs that are tabular alongside imaging to develop models, but it’s really about the pragmatism of deployment at the point of care that hasn’t been feasible until FHIR came about. Ahead, this will iteratively help generate more personalised prognostication, enhance the quality and speed of appropriate referrals and shared care paradigms.”
An example of this is the application of integrated clinical and imaging data with AI, to accurately identify and match potential candidates for clinical trials, accelerating a currently costly and inefficient process, which results in significant drug development delays.
“The opportunity to configure algorithms to different inclusion and exclusion criteria, to reduce screen failures and time to enrolment, presents an exciting prospect, particularly in such an image-reliant specialty such as ophthalmology.”
Dr Kras said now is the time for clinicians to consider the role they will play in this new integrated world, as digital healthcare innovation accelerates to generate sustainable efficiencies and improvement in patient management and healthcare operations.
- Kras A., Oliver W., Gillies M.C., Leng T., et al. Extending interoperability in ophthalmology through Fast Healthcare Interoperability Resource (FHIR). Invest. Ophthalmol. Vis. Sci. 2021;62(8):1708.