Validation of a Deep Learning Algorithm for Diabetic Retinopathy

Background:To validate our deep learning algorithm (DLA) to read diabetic retinopathy (DR) retinographies.

Introduction:Currently DR detection is made by retinography; due to its increasing diabetes mellitus incidence we need to find systems that help us to screen DR.

Materials and Methods:The DLA was built and trained using 88,702 images from EyePACS, 1,748 from Messidor-2, and 19,230 from our own population. For validation a total of 38,339 retinographies from 17,669 patients (obtained from our DR screening databases) were read by a DLA and compared by four senior retina ophthalmologists for detecting any-DR and referable-DR. We determined the values of Cohen’s weighted Kappa (CWK) index, sensitivity (S), specificity (SP), positive predictive value (PPV) and negative predictive value (NPV), and errors type I and II.

Results:The results of the DLA to detect any-DR were: CWK = 0.886 ± 0.004 (95% confidence interval [CI] 0.879–0.894), S = 0.967%, SP = 0.976%, PPV = 0.836%, and NPV = 0.996%. The error type I = 0.024, and the error type II = 0.004. Likewise, the referable-DR results were: CWK = 0.809 (95% CI 0.798–0.819), S = 0.998, SP = 0.968, PPV = 0.701, NPV = 0.928, error type I = 0.032, and error type II = 0.001.

Discussion:Our DLA can be used as a high confidence diagnostic tool to help in DR screening, especially when it might be difficult for ophthalmologists or other professionals to identify. It can identify patients with any-DR and those that should be referred.

Conclusions:The DLA can be valid to aid in screening of DR.