A Real-World Comparison of Three Deep Learning Systems for Diabetic Retinopathy in Remote Australia

Background/objective: Deep learning systems (DLSs) may improve access to screening for diabetic retinopathy (DR), a leading cause of vision loss. Therefore, the aim was to prospectively compare the performance of three DLSs, Google ARDA, Thirona RetCADTM, and EyRIS SELENA+, in the detection of referable DR in a real-world setting. Methods: Participants with self-reported diabetes presented to a mobile facility for DR screening in the remote Pilbara region of Western Australia, which has a high proportion of First Nations people. Sensitivity, specificity, and other performance indicators were calculated for each DLS, compared to grading by an ophthalmologist adjudication panel. Results: Single field colour fundus photographs from 188 eyes of 94 participants (51% male, 70% First Nations Australians, and mean ± SD age of 60.3 ± 12.0 years) were assessed; 39 images had referable DR, 135 had no referable DR, and 14 images were ungradable. The sensitivity/specificity of ARDA was 100% (95% CI: 91.03–100%)/94.81% (89.68–97.47%), RetCAD was 97.37% (86.50–99.53%)/97.01% (92.58–98.83%) and SELENA+ was 91.67% (78.17–97.13%)/80.80% (73.02–86.74%). Conclusions: In a small, real-world service evaluation, comprising majority First Nations people from remote Western Australia, DLSs had high sensitivity and specificity for detecting referable DR. A comparative service evaluation can be useful to highlight differences between DLSs, especially in unique settings or with minority populations.