Telemedicine for Retinopathy of Prematurity

Introduction

Retinopathy of prematurity (ROP) is a disease of the retinal vasculature that remains a leading cause of childhood blindness in the United States and abroad.^1,2 The disease affects the most severely premature and lowest birth weight infants, with 15.6% of premature newborns with hospital stays >28 days and 68% of infants with birth weights <1,251 g affected by the disease.²

Several studies have confirmed the ability of timely treatment to prevent blindness; therefore, a simple, valid, noninvasive, and inexpensive screening examination is necessary to identify infants who are at increased risk for developing ROP.^3–6 The availability of ROP screening is a requirement for level IIIB neonatal intensive care unit (NICU) designation in the United States, which supports infants with extreme prematurity, extremely low birth weights, or severe/complex illnesses.⁷ Although a matter of some debate in the ophthalmic literature,^8,9 the reference standard for the screening and diagnosis of ROP is a live dilated examination by an ophthalmologist using binocular indirect ophthalmoscopy (BIO), often with scleral indentation.¹⁰

However, in many areas, there are a limited number of ophthalmologists and significant workforce limitations, including concerns about medicolegal liability, low reimbursement, and work-flow difficulties, such that few trained ophthalmologists may be available and/or willing to perform ROP screening examinations.^11–13 In addition, the significant variability that exists between examiners diagnosing ROP using BIO also suggests that the use of remote imaging and computer-based image analysis (CBIA) methods may improve accuracy and consistency of diagnosis of plus disease.^11,14–17 A study evaluating ROP image grading by eight ROP experts found that there is poor agreement on the classification of plus disease, despite established international standards.¹⁴ This disagreement suggests that treatment recommendations likely vary among providers and that some infants may be undertreated while others are overtreated for ROP.¹⁴

For these reasons, there has been growing interest in photographic screening and remote interpretation for ROP screening, with reports from several successful clinical implementations and research studies available in the literature. Consequently, the 2013 joint guidelines from the American Academy of Pediatrics, American Academy of Ophthalmology, American Association for Pediatric Ophthalmology and Strabismus, and the American Association of Certified Orthoptists recognized the interest in remote interpretation of retinal images and allowed for the possibility of alternative screening strategies.¹⁸ Current guidelines support the use of remote digital fundus imaging to identify individuals with referral-warranted ROP (RW-ROP), but recommend that at least one BIO examination is completed before initiation of treatment or termination of ROP monitoring, as current cameras do not allow for adequate view of the peripheral retina.^12,19,20 Still, many remain skeptical of the safety and efficacy of telehealth screening as evidenced by a survey of 847 level III NICU directors in which only 21% of NICUs used retinal imaging devices and only 30% agreed that telemedicine for ROP screening is safe.²¹

Background

Reports from the early 21st century documented proof-of-principle for ROP telehealth screening, but raised concerns about the technical ability of the RetCam device (Natus Medical, Inc., Pleasanton, CA) to capture images with sufficient sensitivity to replace live screening.^22,23 However, subsequent reports began to show improved diagnostic capability and low false negative rates, which must be minimized in any ROP screening scenario given the severe consequences of even a single missed case.^24–30

Nevertheless, debate continues and most authors conclude that wide-field imaging could potentially serve as an adjunct to live screening, but ought not replace in-person examination by an ophthalmologist.^12,28,29 A 2008 systematic review likewise concluded “the evidence base is not sufficient to recommend that retinal imaging be routinely adopted by NICUs to identify infants who have serious retinopathy of prematurity.”³¹ Most guidelines continue to be hesitant about ROP telehealth screening and continue to recommend a hybrid approach, given that few large-scale outcome comparisons have been published.²⁰

Recent Clinical Study Findings

The most recent large multicenter validation study to be published, the telemedicine approaches to evaluating acute-phase ROP (e-ROP) Study, compared wide-field retinal imaging performed and interpreted by nonphysicians with examinations performed by physicians.^1,32 The e-ROP Study enrolled 1,257 infants who received a median of 3 imaging sessions and conventional live examinations at 12 sites in the United States and 1 site in Canada between 2011 and 2013. The infants had a median birth weight of 860 g and a median gestational age of 26 weeks. Approximately 44% of infants were nonwhite or did not have race information available. Any ROP was identified in 63.7% of infants and RW-ROP (plus disease, ROP in zone I or stage 3 ROP or greater)²⁵ was noted in 19.4% of infants on criterion-standard live examination. When both eyes were analyzed together (i.e., at the level of the infant), remote grading by trained nonphysician graders had 90% sensitivity and 87% specificity. Given the prevalence of RW-ROP in this population, this conferred a 97.3% negative predictive value and 62.5% positive predictive value. Importantly, when considering only those infants ultimately treated for ROP, the sensitivity of remote imaging grading was 98.2%. Although this number is impressively high, in absolute terms, there were 3 infants out of the 162 treated who did not have RW-ROP detected on the remote imaging preceding treatment.

The e-ROP authors argued that their study supports the validity of using nonphysician imagers and graders for remote detection of RW-ROP, similar to how reading centers are structured for other ophthalmic conditions.^1,33 The authors did highlight the limitations of identifying important features of ROP and inherent variability of the criterion standard live examination as a potential weakness of the study, but also note that the possibility of missing severe ROP needs to be considered in the development of any screening program.^1,34 The e-ROP Cooperative Group found that the region of the retina where most severe disease occurs (zone I) may be best assessed by retinal images, but that the subtleties that may be seen in stage 3 ROP in zone I may currently be best identified by an experienced clinician on a live examination.³⁴ The imaging and informatics in retinopathy of prematurity (i-ROP) consortium also found that there was a slightly higher accuracy for diagnosis of zone III and stage 3 ROP on live examination than with imaging.³⁵ Although examinations by an experienced clinician currently remain the gold standard for ROP screening, there are weaknesses to this system to which the implementation of tele-ROP screening and automated image analysis may be the solution.

Turnaround times of 24 h or less were feasible in the e-ROP study, with >95% of images returned within that period, showing that ROP telemedicine is capable of providing timely feedback for detection of ROP.¹¹ The biggest barriers to rapid turnaround identified were the time of submission and delays between image acquisition and uploading. Images that were submitted before 2 pm were graded much more quickly than images that were submitted later and, therefore, not graded until the next morning. The authors felt these issues could be addressed by improving technology used to select and submit images to allow images to easily be submitted at the bedside and increasing staffing at reading centers during peak demand times.

Reports of Clinical Implementations

United States

A hybrid model has been deployed at six NICUs in Northern California through the Stanford University Network for Diagnosis of Retinopathy of Prematurity (SUNDROP) in which all infants meeting screening criteria were photographed according to screening guidelines and then also receive a live examination within 1 week of NICU discharge. A retrospective analysis of 6 years of follow-up between 2005 and 2011 has been published.² During this time, 1,216 eyes were screened, generating 26,970 retinal images. Twenty-two infants were determined to have treatment-warranted ROP (TW-ROP; zone I, any stage ROP with plus disease; zone I, stage 3 ROP with or without plus disease; zone II, stage 2 or stage 3 ROP with plus disease; any plus disease; or any stage 4 or higher disease).⁴ All TW-ROP infants were successfully identified through photoscreening in this time period and only one “false-positive” case was noted in which stage 3 ROP was not felt to warrant treatment on live examination. These results translate to a sensitivity of 100%, negative predictive value of 100%, specificity of 99.8%, and positive predictive value of 95.5%. The SUNDROP authors concluded that telehealth screening can be safe, reliable, and cost-effective when coupled with committed ROP specialists to interpret images and perform live examinations when necessary.

A similar retrospective “real-world” study from a NICU in Montana reported good outcomes in the 137 infants evaluated, 13 of whom required transfer, and 9 of those transferred ultimately required laser treatment.³⁶ Over the 4.5 years covered by their review, the authors noted no infants progressing to stage 4 or stage 5 ROP. The investigators followed the SUNDROP protocol for their screening schedule and ensured all infants were seen for a live diagnostic examination within 2 weeks of NICU discharge.

International

Vinekar et al. reported results from 36 rural NICUs in the southern Indian state of Karnataka starting in February 2011 through February 2015, covering remote screening of 7,106 infants as part of the Karnataka internet assisted diagnosis of retinopathy of prematurity (KIDROP) program.³⁷ The overall incidence of any ROP was 22.4% and treatment-requiring ROP was 3.6%. In this report, there was no comparison with criterion standard examination. The group’s prior 2014 report examining 1,601 infants³⁸ did compare nonphysician grading of images versus expert live examinations, but did not clearly report the results of this evaluation.

A national network for ROP screening was also developed and implemented in 11 NICUs in Chile.³⁹ Images were taken by trained nonphysician operators using the RetCam Shuttle and were evaluated independently by two ROP experts. Of the 5,263 imaging sessions performed, 4,903 (93%) were considered good or excellent quality with evaluation of ROP possible in 98% of images. In this network, all screening and examinations were performed by telemedicine, with the exception of BIO examinations performed before treatment. Forty-two infants (4%) were referred for treatment and 98% agreement was found between the initial imaging and clinical examinations.

Lorenz et al. reported their 6-year experience with the wide-field remote screening in five NICUs in Germany.⁴⁰ In this study, all 1,222 infants also received live examinations for comparison. The authors report that all 42 cases requiring treatment were successfully identified by telescreening with an acceptable positive predictive value of 82.4%.

In France, implementation of a telemedicine program for ROP screening resulted in an absolute 57.3% increase in the proportion of examinations completed in accordance with American Academy of Pediatrics guidelines, whereas the screening rates in the control group, which continued ROP screening using live examinations, remained unchanged.⁴¹ The average cost of examination in the telemedicine program was slightly more expensive (∼$22) than the standard procedure of transferring infants to a specialized center for examination by a specialist, but the authors projected that this cost would decrease as the number of examinations completed rose.⁴¹

Another French ROP screening program was conducted in Bordeaux between July 2009 and August 2015 and screened 419 infants using the RetCam 120.⁴² They found any ROP in 27.68% of infants. The authors felt that their exclusively telemedicine screening system was successful at identifying ROP, but did not report any data with regard to predictive values.

Skalet et al. performed a feasibility study on 26 babies in Lima, Peru.⁴³ In this study, 95–97% of image sets were judged to be suitable for ROP grading.

Despite lack of full endorsement in current guidelines, remote ROP screening programs are being developed and implemented by many groups around the world.

Guidelines for Imagers

Although the use of nonphysician imagers is the foundation to the widespread use of telemedicine in ROP screening, it is imperative that imagers are appropriately trained and certified to ensure high-quality images. A team of at least two people is recommended for image acquisition: a certified retinal imager to capture images and an NICU nurse to monitor the infant.⁴⁴ Initial education of imagers in the e-ROP study consisted of general training on ROP, premature infants, and image acquisition including positioning infants and maintaining comfort. On site instruction was provided by the camera manufacturer, including hands-on training with the camera and practice with a model eye. Imagers were also trained on image selection, data entry, and export of images.

Certification in the e-ROP study consisted of knowledge assessments and a practical examination that included submitting three bilateral image sets per protocol from infants. Images were evaluated by the reading center and feedback was provided with additional image sets submitted until sufficient quality was obtained. After certification, feedback was provided to sites monthly during calls and yearly at group meetings. The authors found that it was important for imagers to frequently image a varied patient population to maintain optimal skills. The e-ROP study demonstrated a 92% success rate for nonphysician imagers providing acceptable quality images.

In addition, the KIDROP program has developed a 90-day training that is available through an e-learning platform “WISE-ROP.”^®45 Imagers read modules and complete quizzes to evaluate their progress. Video sessions and oral trainings are used to discuss the imagers’ technique and hands-on sessions are scheduled with an assigned mentor.

A rigorous training and certification program is necessary for implementation of telemedicine in ROP screening to ensure high-quality images are consistently acquired.

Imaging Systems

One of the major considerations in any ROP telemedicine program is the choice of digital imaging system. Until 2016, most reports on ROP telehealth programs used the RetCam^® system.^1,28 There are now several wide-field contact imaging systems on the market, although none have published clinical validation studies. The Visunex Panocam^® (Visunex Medical Systems, Inc., Fremont, CA) system has two cameras in its product line,⁴⁶ a smaller portable system and a larger console system. The Phoenix ICON^® system, a contact wide-field cart-based system (Phoenix Technology Group, Pleasanton, CA), has also recently been introduced.⁴⁷ The 3Nethra Neo^® (Forus Health, Bangalore, India), a 120° field of view, contact camera, has also recently been introduced.⁴⁵ In a small pilot study of 128 premature infants from 35 NICUs, images acquired by both the Neo and RetCam were evaluated by two masked ROP specialists.⁴⁸ The Neo was reported to have sensitivities of 97.4% and 99.3% and specificities of 81.1% and 75.6% for each grader, respectively. Since initial reporting, the study has been expanded to include 1,200 infants, but results are not yet published.

The Pictor^® (Volk Optical, Inc., Mentor, OH), a handheld noncontact fundus camera, has also been shown to be effective in screening for type 1 ROP and preplus and plus disease, despite its 45° field of view.^49,50 The Pictor was found to have 100% sensitivity by both graders and 93% and 74% specificity by each grader, respectively, when compared with clinical examinations.⁵⁰ At ∼$10,000, the Pictor may make implementation of telemedicine ROP screening programs more widely accessible.⁵⁰

With the introduction of new cameras to the commercial market, investigators have found entry prices to be 40–50% of the recent past prices,⁵¹ making remote ROP screening systems more widely accessible.

Guidelines for Reading Centers

Although current guidelines recommend that graders for telemedicine ROP screening programs be experienced ophthalmologists who have experience in bedside examination as well as interpretation of digital images, several studies have examined the efficacy of the use of nonphysician graders.²⁰ Nonphysician graders in the e-ROP study underwent a three-phase process including training, precertification, and final certification.⁵² Phase 1 of training included lectures that covered classification of ROP, the study and grading protocol, and current ROP treatments, interactive sessions with sample images, and a visit to an NICU to observe the imaging process. To progress to phase 2, graders were required to pass a knowledge assessment. Phase 2 included grading of an average of 15 image sets along with review and discussion of the results compared with an expert consensus generated final result. Phase 3 included grading of 100 ROP training image sets with additional images added until 85% agreement was met. Final certification consisted of 15 image sets from the e-ROP pilot submission and was earned once 80% agreement was met. If this level of agreement was not achieved, retraining was performed for 1 week and the final certification with new images was repeated. This process was repeated until 80% agreement was met. After using this system, the authors reported a weighted kappa of 0.72 for intergrader agreement for RW-ROP as well as weighted kappas ranging from 0.57 to 0.94 for intragrader agreement for RW-ROP.

Despite the current guideline recommendations for physicians to evaluate digital images, there is inconsistent training on ROP and no standardized method of assessing competency among ophthalmology residency and pediatric ophthalmology and retina fellowship programs.⁵³ The Global Education Network-ROP group has created a tele-education program for ROP to further the education of physicians evaluating ROP images. The program includes a pretest, ROP tutorial on classification and management, five training chapters that each emphasize a particular category of ROP, and a post-test. This education system has been studied in two separate populations: 31 ophthalmology residents among 5 residency programs in the United States and 1 residency program in Canada and 58 ophthalmology residents and fellows from 1 program in Mexico.^53,54 Both studies found that the system was effective in improving diagnostic accuracy of ROP by ophthalmologists-in-training. Although this program has limitations, such as not tracking common errors made by trainees and not including examples of stage 4 or stage 5 ROP, the authors feel that improvements can be made and that this platform has potential to be used in a widespread manner to standardize evaluation of ROP images for both physician and nonphysician graders. Appropriate training of both physician and nonphysician graders is essential to ensure patient safety.

Automated Image Analysis

In concert with early reports of successful application of retinal photographic screening for ROP, interest in automated image analysis for image interpretation was also evident. Several early groups sought to determine whether the vascular tortuosity of plus disease could be segmented in an automated or semiautomated manner.^55,56

Subsequently, several groups focused on integrated grading systems. Ataer-Cansizoglu et al., who are part of the i-ROP consortium, reported on their validation study in which 77 wide-angle images were graded by a computer algorithm “developed to extract tortuosity and dilation features from arteries and veins.”⁵⁷ The algorithm grades were compared with a reference standard diagnosis generated by combining three independent expert image grades with the diagnosis rendered during a live BIO examination. The investigators found that their system was 95% accurate for the classification of preplus and plus disease, which compared favorably with the individual accuracy of the expert grades and was substantially higher than the mean accuracy of 31 nonexperts.

The i-ROP consortium also evaluated the methods physicians currently use when diagnosing ROP to further understand what may work best in an automated system. They found that ROP experts consider tortuosity of both arteries and veins and also consider areas outside the central retina when diagnosing plus disease, contrary to the International Classification of Retinopathy of Prematurity standards.¹⁶ They found that the performance of the i-ROP CBIA performed better than 9 of 11 experts in the study with 95% accuracy for diagnosis of plus disease when using a larger field of view than recommended and considering all vessels.¹⁶

Abbey et al. have also recently reported their validation of the ROPtool system.⁵⁸ For this study, 335 fundus photos were collaboratively assessed by a panel of 3 ROP experts to generate the criterion standard grade. Each quadrant was graded on a 5-point scale that incorporated tortuosity and dilatation. If any quadrant was graded as questionable or worse, then the image was classified as abnormal. The ROPtool system calculates the tortuosity of a single vessel within each quadrant, and the value for the second most tortuous segment is defined as the tortuosity score for that eye. Dilation was also assessed, but this did not improve their model accuracy and was not presented. The authors then examined multiple diagnostic set points through receiver operating characteristic. Optimizing sensitivity and including unreadable images as diseased, ROPtool had a sensitivity of 96% and specificity of 64%. The clinical utility of the proxy of tortuosity used as the criterion standard for this validation was not discussed.

More recently, deep learning, where CBIA systems have been trained to automatically recognize and evaluate images, has been used for ROP screening.^59,60 Deep learning allows the system to continually learn and re-evaluate its process autonomously and consists of multiple layers of algorithms that data flow through to form a neural networks.⁶⁰ Convolutional neural networks have to be trained through exposure to a large number and variety of pathological and normal images to then apply a series of filters to produce the desired output, which in this case would be diagnosis or classification of ROP.^60,61

The i-ROP consortium has developed a deep learning algorithm, which has shown a high accuracy for identifying plus disease.⁶¹ The system was trained using a set of 5,511 retinal images that had been obtained as part of the i-ROP study and a reference standard diagnosis established by three trained graders and one expert clinical examiner. The system was able to diagnose plus disease on an independent set of 100 images with 93% sensitivity and 94% specificity and preplus disease or worse with 100% sensitivity and 94% specificity. The algorithm outperformed six of eight ROP experts and all prior computer-based imaging analysis systems in ROP without the need for manual segmentation.⁶¹ After the algorithm was trained to recognize plus disease, the authors also tested its ability to identify diagnostic categories and overall disease severity. After analysis of 4,861 images, they found that the system could accurately detect clinically significant ROP with 94% sensitivity and a 99.7% negative predictive value based on posterior pole fundus photographs alone.⁵⁹

Wang et al. also developed two deep neural networks, Id-Net and Gr-Net, which were, respectively, designed for the identification and grading of ROP.⁶² Id-Net achieved a sensitivity of 96.62% and specificity of 99.32% for identification of any ROP and Gr-Net achieved 88.46% sensitivity and 92.31% specificity for grading of ROP severity, which was comparable with three expert graders.

Zhang et al. have also evaluated three general-purpose deep neural networks (AlexNet, GoogLeNet, and VGG16) using a transfer learning workflow with 17,801 images to identify ROP.⁶³ They found that VGG16 achieved the best performance on a test set of 1,742 images and found that this performance was comparable with that of five pediatric ophthalmologists.

The use of CBIA systems in ROP screening could help improve the accuracy and consistency of diagnosis of ROP.

Safety of Retinal Imaging in ROP

Although many remain skeptical of the safety of remote ROP imaging and grading of images,²¹ several studies have reported low frequencies of adverse events (AEs) associated with retinal imaging. In the e-ROP study, one-third of AEs were reported to have probably or definitely been related to BIO (4 AEs) or contact imaging (18 AEs).⁶⁴ Based on the low frequency of AEs (65 AEs reported >4,238 visits) and serious AEs (none) reported in the e-ROP study, the authors considered both BIO and imaging to be safe methods of ROP screening.⁶⁴

Prakalapakorn et al., who have examined the use of the Pictor noncontact fundus camera, also found that safety events (clinically significant bradycardia, tachycardia, oxygen desaturation, or apnea) occurred after 5.8% of clinical examinations and after 0.8% of imaging sessions.⁶⁵ Because the noncontact camera did not require a use of a lid speculum or contact with the cornea, the authors felt that the process was less stressful for infants.

Despite the survey finding in which only 30% of NICU directors felt that telemedicine for ROP screening was safe, studies evaluating AEs surrounding the use of ROP imaging have so far found low incidences of AEs.

Costs of Remote ROP Screening

A major barrier to implementation of telemedicine in general is the high startup cost. Within ocular telehealth, the retinal cameras needed for imaging premature infants are costlier than the nonmydriatic devices used to image adults with diabetes, but costs are decreasing with the release of new cameras, such as the Neo, and expansion to nontraditional cameras, such as the Pictor.

Several cost-effectiveness analyses have been published exploring different scenarios for ROP screening. Jackson et al. performed a cost–utility analysis of telemedicine and standard ophthalmoscopy compared with no treatment from a third-party perspective.⁶⁶ This group found that the cost per quality-adjusted life year (QALY) gained was $3,193 for telehealth screening compared with $5,617 with standard ophthalmoscopy. Varying several aspects within the simulation generated wide variations from their base case (up to $18,989 per QALY gained for telehealth and $27,215 for ophthalmoscopy), but the interventions remained below the previously described threshold of a highly cost-effective intervention of $50,000/QALY. Because the perspective chosen for this analysis (third-party payer) does not include the costs of acquiring the retina cameras and telehealth connectivity, the results are valuable in convincing policy-makers and insurers of the value of the intervention, but do not necessarily speak of the viability of establishing a telehealth program for hospitals.

Castillo-Riquelme et al. likewise performed a cost-effectiveness of retinal photographs screening for ROP in the United Kingdom.⁶⁷ This simulation study compared five different screening strategies and used a health system perspective. The investigators estimated that the current methods cost GBP 321 to screen one infant, and that if a specialist nurse were to travel among NICUs to capture and interpret images, this would be substantially less expensive (GBP 172 per infant or GBP 201 if the images were transmitted for ophthalmologist review). Other methods explored in their simulation would be more expensive: use of a standard camera with NICU nurses acquiring and interpreting the images (GBP 371) or transmitting the images for ophthalmologist review (GBP 390). Throughout the sensitivity analysis, the least expensive method was largely unchanged, unless the cost of the visiting nurse was almost at the extreme high end of the sensitivity range or the specificity of nurse interpretation was 40% or below (99% was used in the base case). Of note, if the sensitivity dipped slightly below 90%, the standard examination strategy was noted to be “cost-effective.” The authors suggest that development of a portable imaging solution could dramatically change the cost-effectiveness landscape. The results may be difficult to apply to other settings without a national health system.

Makkar et al. noted that implementing telemedicine examinations for ROP in a level II NICU reduced costs associated with transport, decreased the length of hospitalization, and decreased the use of higher levels of care than needed.⁶⁸ They also noted that the current telemedicine reimbursement rate for digital retinal examinations does not cover the cost of required effort (∼1 h of total processing time for each infant imaged).

Conclusions

As the preceding discussion illustrates, the ocular telehealth paradigm for ROP is different from remote screening for diabetic retinopathy (DR) as discussed in the current edition of the American Telemedicine Association (ATA) Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy.⁶⁹ The population at risk is hospitalized low-birth weight premature infants, so the technical aspects of image acquisition need to account for the NICU environment and the anatomy of the neonatal eye. For ROP, the burden of screening largely falls on the providers and health systems rather than on patients to present for opportunistic screening. Perhaps the most critical difference is the time course of vision loss and prognosis for eyes in which the ROP diagnosis is not made in a timely manner. Unlike in DR, very high sensitivities for vision-threatening disease must be achieved because the risk of near-term potentially permanent vision loss is unacceptably high.

In addition to the challenges posed by ROP for effective ocular telehealth, there are opportunities that are unique to ROP. Because the screenings are done in a controlled environment, universal coverage of screening should be far easier to achieve than in DR, and technical issues with equipment may be easier to deal with in the NICU environment with access to hospital IT and biomedical engineering departments. DR telehealth screening programs sometimes are met with resistance from primary eye care providers as the programs can be seen as a threat to patient volumes and revenue streams. ROP screenings, in contrast, are often a challenge for hospitals to find appropriate coverage and could provide more convenience to treating providers rather than “competition.” Finally, there may be medical–legal benefits to photodocumentation of ROP examination findings, particularly if there are automated aids to image classification or decision support within the grading software.

Any ROP screening implementation using telehealth should follow the screening recommendations of the major societies of the region.¹⁸ Retinal images must be of sufficient quality to allow a grader to make an accurate determination of the ROP status. Different groups have used different diagnostic set points as already discussed, so any program must validate to their predetermined level of disease severity, analogous to the recommendations made in the 3rd Edition of the ATA Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy.⁶⁹ The majority of research to date has used contact wide-field imaging, but research is ongoing to determine the value of noncontact posterior pole imaging to detect plus disease to screen for RW-ROP.⁷⁰ Because the volume of babies requiring imaging in a given center may be lower than what is seen in DR screening, deliberate efforts should be taken so that imagers maintain skills. Further technical guidance is provided in a 2015 Joint Technical Report of the American Academy of Pediatrics, the American Academy of Ophthalmology, and the American Association of Certified Orthoptists.¹²

Disclosure Statement

J.P.C. receives grant support from Genentech, and is listed on a preliminary patent application related to deep learning technology for ROP screening.

Funding Information

C.J.B. was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number P20GM103644. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. S.D. was supported in part by the Elliot W. Shipman Professorship Fund. J.P.C. was supported by National Institutes of Health grants R01EY19474, P30EY10572, and K12EY27720 (Bethesda, MD), National Science Foundation grant SCH-1622679 (Arlington, VA), and unrestricted departmental funding and a Career Development Award (JPC) from Research to Prevent Blindness (New York, NY).