BACKGROUND: Community-based optometrists, a major provider of primary eye care in the United Kingdom, are the main source of referrals to hospital eye services. The widespread introduction of optical coherence tomography devices in community practices provides community-based optometrists with an opportunity to identify a broader range of treatable diseases. Standard referral pathways do not effectively filter unnecessary referrals, with misclassification of urgency, and erroneous diagnoses.
OBJECTIVES: To assess the effectiveness of a teleophthalmology referral pathway between community-based optometrists and hospital eye services for retinal diseases. To measure the accuracy of an artificial intelligence decision support system for diagnosis and referral management of retinal disease.
DESIGN: A multicentre, superiority cluster randomised controlled trial to assess the effectiveness of a teleophthalmology referral pathway. A prospective, observational diagnostic accuracy study to measure the performance of artificial intelligence decision support system. A comprehensive economic evaluation was conducted.
SETTINGS: United Kingdom-based community optometry practices with an optical coherence tomography device and hospital eye services.
PARTICIPANTS: Adults requiring referral for retinal disease at the opinion of the community-based optometrists.
INTERVENTIONS: Community optometry practices were randomised 1 : 1 to standard care or teleophthalmology. Referrals sent via the teleophthalmology platform were remotely reviewed by human experts based at the corresponding hospital eye services. A referral decision was provided within 48 hours. Suitable optical coherence tomography scans were solely processed by artificial intelligence decision support system (the 'Octane' model).
MAIN OUTCOME MEASURES: Cluster randomised controlled trial's primary outcome was the proportion of false-positive referrals (not required or not urgent) per arm in overall participants and in referred-only participants against an independent reference standard. Secondary outcomes included the proportion of wrong diagnosis, wrong referral urgency, false-negative referrals, safely triaged referrals for rare diseases, time from referral to consultation and treatment and cost-effectiveness of teleophthalmology. Primary outcome for the artificial intelligence study was the sensitivity and specificity of artificial intelligence referral decisions against the reference standard.
RESULTS: Teleophthalmology significantly reduces the proportion of false-positive urgent referrals by 59% compared to standard care in referred participants. Due to the observed low event rate for false positive referrals, teleophthalmology's role for reducing false positives overall was inconclusive. No significant difference between arms for safety of referral decisions (false negatives) was found. After accounting for external factors, the time to consultation demonstrated both clinically and statistically significant benefits for the teleophthalmology arm. The time to treatment showed a clinically significant benefit. Of 396 recruited participants, the Octane artificial intelligence model processed images contributed by 204 participants (51.5%). For referral decisions, the model showed comparable sensitivity and specificity against its own preset referral rules (rule-based reference standard) (post hoc analysis), but it showed inferior sensitivity and specificity when compared to human expert assessors making these referral decisions (clinical reference standard) (primary AI analysis). The artificial intelligence model presented challenges relating to its generalisability in a real-world evaluation context.
LIMITATIONS: Technical limitations in optometry practices, lack of ethnicity data.
CONCLUSIONS: Asynchronous teleophthalmology reduces the number of unnecessary urgent referrals, the main drivers of increasing hospital capacity pressures, provides more appropriate referral-to-treatment times and is more cost-effective compared to standard care. The Octane artificial intelligence model could not process images from 48.5% of study participants. Compared to hospital-based experts for referral decisions, Octane was less accurate at making routine and urgent referral decisions and of similar accuracy to community optometrists.
FUTURE WORK: Applied health research, human-artificial intelligence interaction and artificial intelligence clinical trial design.
TRIAL REGISTRATION: This trial is registered as ISRCTN18106677.
FUNDING: This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: NIHR127773) and is published in full in; Vol. 29, No. 69. See the NIHR Funding and Awards website for further award information. Health Technology Assessment
