Dr. Khandekar Rajiv, K9599.
Running title:RETeval and DR screening
Authors:
Rajiv Khandekar MS (Ophth) PG Dip (Epi)2
Humoud M. AlotaibiMD1
M.D. AlotaibiMD1
Ches SouruBSc (nursing)2
Donald U. StoneMD2,3
Igor. KozakMD2
Affiliations
1King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia,
2King Khaled Eye Specialist Hospital, Riyadh, Saudi Arabia
3Wilmer Eye Institute, Baltimore, Maryland, United States
Introduction
In 2014, 387 million people were estimated to be living with diabetes, an alarming number that is set to rise to 592 million within the next twenty years.1 Diabetic retinopathy (DR) needs public health approach in coming years to prevent visual disabilities.2 Early detection and prompt action have been found to delay development as well as progression of diabetic retinopathy.3 Unfortunately, many cases of diabetic retinopathy are detected with symptoms and in late stages and hence annual DR screening of patients with type II diabetes is recommended.4 The existing work force is unable to undertake annual DR screening and hence innovative manners of DR screening are recommended. Telemedicine and transfer of digital images of retina is one of the gold standards for diagnosing different stages of diabetic retinopathy from remote areas.5 However, action oriented DR screening will be justified for underprivileged and developing countries.
The cost and portability of digital fundus camera limits its application for DR screening. Although a number of models for mobile unit of DR screening are applied, the coverage is limited in large countries like India and China.6-8
A hand held portable equipment that can be used by non-ophthalmic person to detect DR through undilated pupil will be welcome by public health authorities dealing with visual disabilities due to diabetic retinopathy.
Different components of retinal function including electrophysiologic responses are altered at different stages of DR. The electroretinography (ERG)full-field flicker systems measure response from the cone system and are representative of the whole retina. They are useful mainly in differentiating sight threatening DR from non-proliferative stages.9,10
A new handheld electrophysiological flicker ERG recording device – RETeval (LKC Technologies, Inc. Gaithersburg, MD, Welch Allyn, Inc. Skaneateles Falls, NY, U.S.A.)has been developed.11It simultaneously measures the full-field flicker ERG and pupillary response to light. It has been shown to perform well in non-mydriatic eyes12 and cataracts of less than Grade 2.13To the best of our knowledge, only one study is published to validate the DR screening using this new tool.14 The purpose of his study was to evaluate the RETevalas a screening tool in diabetic retinopathy and sight threatening diabetic retinopathy in comparison to conventional digital retinal photographs. We further evaluated usefulness and cost effectiveness of DR screening using RETeval. We also studied added value of Amsler’s central field of vision chart in enhancing the validity of DR screening.
Methods
Subjects
This cross-sectional, single-site, non-interventional study of consecutive patients with diabetes mellitus was undertaken at screening clinic of the King Khaled Eye Specialist Hospital (KKESH) – a tertiary eye care hospital of central Saudi Arabia. The Institutional Research Board approved this study (#1497-P) and the study adhered to the tenets of the Declaration of Helsinki. The study was undertaken between January 2015 and April 2016. Diabetic patients with eye problem(s) seeking ophthalmic advice in our hospital were study population. Written informed consent was obtained from all patients to participate in this research project. Personnel trained in fundus photography and the use of RETevaldevice were the testers. The eyes with severe corneal opacities and cataracts grade 2 and above were excluded from the study as well as eyes with poor quality of retinal image obtained by fundus camera.
At the time of enrolment, we obtained demographic and diabetes related information like age, sex, duration of diabetes, glycemic control based on last HbA1c report and the year of last diabetic retinopathy screening.
Testing
We used non-mydriatic digital funds camera (TRC NW 300, Topcon, USA) to obtain three images (macula including the optic disc, nasal and temporal). The images were reviewed by retina specialist to grade the diabetic retinopathy who was masked to ERG results. The grading followedthe guidelines of American Academy of Ophthalmology.15Wefurther grouped the severe non-proliferative diabetic retinopathy (sNPDR), proliferative diabetic retinopathy (PDR) and diabetic macular edema(DME) together as sight threatening diabetic retinopathy (STDR). The time taken for obtaining the digital images was also noted.
Technical description of RETeval™ (LKC Technologies, Inc. Gaithersburg, MD, Welch Allyn®, NY, USA) device has been published previously.12-14We used following steps for DR screening using RETeval™ (Figure 1).16 We applied sensor strip on check on lateral half of lower lid. Patient was instructed not to talk during the test. The spectacles of the patient were removed. We connected the lead to the sensor strip. The machine was switched on and with fellow eye covered patient was asked to focus on red beam projected form the device. The time from start to the result displayed on the LCD was noted. Form the display information on test pass or fail, amplitude of the wave was noted. Right eye was tested first followed by left eye. Testing took place in room with no background light whichhas been shown to impair detection of STDR.17The sensor strips were disposed to avoid rescreening of other patient using the used strips. The calibration phase followed prior reports with this device.14Based on 20mV amplitude with a constant implicit time, as fixed by manufacturer, the machine was set to declare the test as pass (≥20mV) or failed (<20mV). We also used amplitude with a constant implicit time, as fixed by manufacturer, as a continuous outcome variable to study the validity of RETevalby plotting separate curve for eyes with sight threatening diabetic retinopathy as identified by retina specialist using digital images.
To calculate validity of RETevaloutcomes to the gold standard [interpretation of digital retinal image by retina specialist to grade diabetic retinopathy] we used ‘pass’ and ‘fail’ by RETeval to presence and absence of sight threatening diabetic retinopathy by retina specialist after critically looking at digital fundus images of the retina.
We then performed central vision testing using reverse Amsler grid test. The abnormality noted was defined as ‘fail’test while no abnormality perceived in viewing the grid was considered as ‘pass’ test. The time taken to perform this test was also noted.
Outcomes
The primary outcome of the study was assessment of validity of the instrument as a screening tool in diabetic retinopathy and sight threatening diabetic retinopathy along with assessment of instrument’s sensitivity, specificity, positive and negative predictive values and detection of false positives/negatives. This was studied for instrument alone and in combination with Amsler grid testing to detect sight threatening macular edema. We also evaluated usefulness and cost-effectiveness of this instrument in DR screening.
Samples size calculation
We assumed that the sensitivity of RETeval in detecting STDR is 83% compared to digital photography.14To have 5% error margin, 95% confidence interval of a cross sectional validity study with a clustering effect of 1.5, we needed 326 diabetic persons to include in our study. In view of possible media opacities that prevent us to assess the DR stage, we increased the sample by 33% and the final sample was planned as 400 eyes of 400 diabetic persons. Only one eye with worse stage of DR was included in the study.
Statistics
The data was computed in Microsoft XL® spreadsheet. It was converted into a spreadsheet of Statistical Package for Social Studies (SPSS 22) (IBM, Chicago, USA). For qualitative outcomes like pass and failed test, we calculatedthe frequencies and the percentage proportions. For continuous variables like amplitude of ERG, we calculated the mean and standard deviation provided the distribution of the variable was normal. The validity of the STDR screening and DR screening by RETeval was assessed by comparingits results to the outcomes of retina specialists evaluation of presence and absence of STDR and DR with different stages of DR. The sensitivity, specificity, positive predictive values, negative predictive values of DR screening by RETevalwere calculated. The amplitude of the ERG was also used as continuous outcome variable and Area under curve was plotted and calculated to determine the test validity both for STDR and DR screening. We added outcomes of Amsler screen test into the respective pool of RETeval screening result to calculate revised validity of sequential screening by two methods.18The independent factors like duration of diabetes, gender, age and glycaemic control. We calculated 95% confidence intervals of different validity parameters. We also used two sided p-values for statistical validation.
Results
Subjects
We included 400 eyes with worse stage of DR of 400 diabetic patients in this study. Their demographic and diabetic profile is given in Table1. Theglycemic control was poor in two thirds of diabetic patients (n=293, 73%). The median duration of diabetes was >10 years. Almost two thirds of them had undergone diabetic retinopathy screening at least once in the past (n=243, 61%).
Testing
There were no technical failures in any of the eyes tested. Table 2shows the magnitudeof different stages of diabetic retinopathy per eye and in worse eye based on digital funds camera picture interpreted by retina specialist. The prevalence of diabetic retinopathy of any grade was 48.8% (95% CI 43.9 – 53.7) while the prevalence of STDR was 27% (95% CI 22.6 – 31.4).
The outcome of RETeval test was ‘fail’ (based on 20 µV or more amplitude of electrophysiological spikes)in 351(87.8%; 95% CI 84.5 – 91.0) patients. The amplitude of electrophysiological spikes (µV) as measured by RETeval in eyes with worse stage of DR of diabetic patients by different grades of diabetic retinopathy is presented in Figure 2. If ‘fail’test was defined as 27µV or more amplitude of electrophysiological spikes, all cases of severe NPDR, PDR, DME and STDR would be picked up as failed test.
Validity
Table 3 presents validity parameters of STDR screening by RETeval.With high sensitivity (95.4%) but low specificity (17.5%) of this screening tool, it seems to be a good first level screening tool. We added Amsler grid testing outcomes in the validity parameters of RETevalSTDR screening. There were 34 cases who failed Amsler test among 108 cases of STDR based on digital fundus photo evaluation. There were 44 cases who failed Amsler test among 190 cases without STDR based on the digital fundus photo evaluation. Thus sensitivity of sequential two testing (RETeval and Amsler grid test) was 30.1% and specificity was 80.1%.
Table 4 presents validity parameters of overall DR screening by RETeval. With high positive predictive value (60.7%) of DR screening by RETeval, this test seems to be very useful in population with high prevalence of diabetic retinopathy. The area under receiving operating curve for those diabetic who had both RETeval and digital fundus outcome as positive or negative were plotted for STDR, DR and PDR + SNPDR. The ROC were 76.6%, 50.6% and 74.4% respectively (Figures 3-5).
Usefulness
RETeval was preferred method of DR screening among 250 (63.5%) diabetic patients whereas 91 (22.8%) diabetics preferred digital fundus photography as preferred method of DR screening. Fifty nine (14.5%) patients could not decide their preference for DR screening procedures. In as many as 56 (14%) cases that could not be evaluated using digital fundus camera due to media opacities, poor cooperation or small pupil, RETeval was useful in DR screening. Of the 156 cases with false positives, we could review presence of other posterior segment pathologies in 119 (76%) cases. Seventeen (15%) of them had glaucoma, healed chorioretinitis, optic atrophy and retinal dystrophies.
Testing time
The mean time of testing DR status of two eyes of a diabetic person using RETeval was 5.3 ±2.1 minutes. This was much lower than median 15 minutes (25% quartile of 10 minutes) by digital photograph method of screening.
Cost-effectiveness
The capital cost of RETeval was US $ 4,000 while the recurrent cost of 400 strips for testing diabetics was US $ 4,800. The time spent by technician in undertaking this test was 10 minutes per patient. The capital cost of digital fundus camera was US $ 29,333. The time spent by medical student to take retinal images was 20 minutes per person and retina specialist took one minuteto evaluate six images (three of each eye) of a diabetic person.
Discussion
To our best knowledge, this is the first study to independently investigate the usefulness of RETeval for DR screening with evidence of validity, patient acceptance and cost effectiveness compared to the conventional screening method using color fundus photography. Our study demonstrates that RETeval portable ERG device is an excellent first level diabetic retinopathy screening tool with high sensitivity of disease detection. This compares with previously published report which has compared the device with ETDRS 7-standard field photography.14 Additionally, we report that it is easy to use,is patient friendly, faster and seems to be cost-effective.This is the first study to use this device in the Middle Eastern diabetic population.
As a stand-alone test we detected its high sensitivity and low specificity for DR. We attribute low specificity to higher rate of false positives which occurs with screening devices. Of the false positives detected by RETeval screening, 15% included other posterior segment pathologies in addition early stages of DR. For comparison, frequency doubling technology (FDT) in glaucoma screening has been noted to have good sensitivity but with high rate of false positives.19Similarly among population with diabetes, FDT has been demonstrated to be good not only for glaucoma detection but other posterior segment pathologies including DR.20Similarly, outcomes of RETevalscreening for STDR may be contaminated by the presence of other pathologies which increase its sensitivity but do not add to specificity of the disease we are detecting.
By combining ERG screening with Amslergrid test, the sensitivity of the test declined and specificity increased. Amsler grid detects macular pathology, namely diabetic macular edema as part of STDR. We have followed manufacturer’s recommendations and prior report14 to set a cutoff value for above which subjects are considered to have tested positive. Our subanalysis shows that if the equipment is set (higher) for 27MV amplitude as pass/fail parameter, the sight threatening diabetic retinopathy could be detected more effectively thus increasing test specificity.
The digital images demonstrate anatomical derangement of retinal tissue due to diabetic retinopathy while RETevalscreening outcomes provide information on functional limitations of retina due to effects of diabetic retinopathy. Thus, outcomes of both DR screening methods could be considered as complimentary to each other. Compared to the previous report14 we found no technical failure during screening process in any of the eyes. It performed well in non-mydriatic eyes and eyes with early cataracts delivering stimulus of constant retinal luminance. We have not calculated reproducibility of ERG testing in our cohort. However, prior study using the same instrument has shown high intraclass correlation of measurements with the device.14
Although the capital cost of this DR screening procedure is low, recurrent cost could pose problem for accepting it as universal screening procedure in developing countries. However, it could be used by non-ophthalmic personnel at diabetes centers or primary health centers where diabetic patients visit their physician periodically. A feasibility study to undertake DR screening using RETevalat primary health center by family physician is further recommended.
While the study was performed at the tertiary eye care facility, the testing itself was done at the screening clinic which decides eligibility for tertiary care. We, therefore, believe that the outcomes of the study can be applicable to the general population in the Middle East.
In summary, we find RETevaldevice a good, efficient and cost-effective first line tool to screen for DR in population with diabetes mellitus.
References
- International Diabetes Federation. A huge and growing problem in Annual Report 2014. http://www.idf.org/sites/default/files/IDF-2014-Annual-Report-final.pdf last accessed on 1st May 2016.
- Ting DS, Cheung GC, Wong TY. Diabetic retinopathy: global prevalence, majorrisk factors, screening practices and public health challenges: a review. ClinExperiment Ophthalmol. 2015 Dec 30. doi: 10.1111/ceo.12696.
- Ding J, Wong TY. Current epidemiology of diabetic retinopathy and diabeticmacular edema. CurrDiab Rep. 2012;12(4):346-54.
- Scanlon PH, Aldington SJ, Stratton IM. Epidemiological issues in diabeticretinopathy. Middle East Afr J Ophthalmol. 2013;20(4):293-300.
- Zimmer-Galler IE, Kimura AE, Gupta S. Diabetic retinopathy screening and theuse of telemedicine. CurrOpinOphthalmol. 2015;26(3):167-72.
- Das T, Raman R, Ramasamy K, Rani PK. Telemedicine in diabetic retinopathy:current status and future directions. Middle East Afr J Ophthalmol. 2015;22(2):174-8.
- Shi L, Wu H, Dong J, Jiang K, Lu X, Shi J. Telemedicine for detecting diabeticretinopathy: a systematic review and meta-analysis. Br J Ophthalmol. 2015;99(6):823-31.
- Waugh NR, Shyangdan D, Taylor-Phillips S, Suri G, Hall B. Screening for type 2diabetes: a short report for the National Screening Committee. Health TechnolAssess. 2013;17(35):1-90.
- Pescosolido N, Barbato A, Stefanucci A, Buomprisco G. Role ofelectrophysiology in the early diagnosis and follow-up of diabetic retinopathy. JDiabetes Res. 2015;2015:319692.
- Bresnick GH, Palta M. Temporal aspects of the electroretinogram in diabetic retinopathy. Arch Ophthalmol 1987;105:660-664.
- RETeval device a viable option for diabetic retinopathy screening. Highlight from ADA 2014. In Healio Ocular surgery News. http://www.healio.com/ophthalmology/retina-vitreous/news/online/%7B394c4f73-10e0-4258-83dd-ff970fe59317%7D/reteval-device-a-viable-option-for-diabetic-retinopathy-screening last accessed on 1st May 2016.
- Kato K, Kondo M, Sugimoto M, Ikesugi K, Matsubara H. Effect of pupil size on flicker ERGs recorded with RETevalsystem: new mydriasis-free full-field ERG system. Invest Ophthalmol Vis Sci 2015;56:3684-3690. DOI:10.1167/iovs.14-16349.
- Miura G, Nakamura Y, Sato E, Yamamoto S. Effects of cataracts on flicker electriretinograms recorded with RETeval™ system: new mydriasis free ERG device. BMC Ophthalmol 2016
- Maa AY, Feuer WJ, Davis CQ, Pillow EK, Brown TD, Caywood RM, Chasan JE,Fransen SR. A novel device for accurate and efficient testing forvision-threatening diabetic retinopathy. J Diabetes Complications. 2016;30(3):524-32.
- American Academy of Ophthalmology. International Clinical Classification for Diabetic retinopathy and Diabetic Macular Edema – 2012. http://www.aao.org/clinical-statement/international-clinical-classification-system-diabe last accessed on 2nd May 2016.
- Welch Allyn®RETeval-DR™. Operating instructions in Directions for use. 2015;19-28. https://www.welchallyn.com/content/dam/welchallyn/documents/sap-documents/LIT/80020/80020080LITPDF.pdf last accessed on 2nd May 2016.
- Tyrberg M, Lindblad U, Melander A, Lovestam-Adrian M, Ponjavic V, Andreasson S. Electrophysiological studies in newly onset type 2 diabetes without visible vascular retinopathy. Doc Ophthalmol 2011;123:193-198.
- Kanchanaraksa S. Evaluation of Diagnostic and Screening Tests:Validity and Reliability. The Johns Hopkins University. 2008. http://ocw.jhsph.edu/courses/fundepi/pdfs/lecture11.pdf last accessed on 2/5/2016.
- Geimer SA. Glaucoma diagnostics. ActaOphthalmol. 2013;91 Thesis 1:1-32.
- Khandekar R, Zutshi R, Ali M, Raisi AA, Dass H. Influence of diabetes on thevalidity glaucoma screening by frequency doubling perimetry: a hospital-basedstudy in Oman. Diabetes TechnolTher. 2008;10(4):278-82.
Figure 1. Picture of electrophysiological tool RETeval (LKC Technologies, Inc. Gaithersburg, MD, Welch Allyn, Inc. Skaneateles Falls, NY, U.S.A.) for diabetic retinopathy screening.
Figure 2. Amplitude of electroretinogram (ERG) spikes as displayed by RETeval while screening of eyes with different stages of diabetic retinopathy.In x axis different stages of diabetic retinopathy are given while in Y axis shows amplitude of elecrophysiological spike (mV) as measured by RETeval.
Figure 3. The Receiver Operating Characteristic (ROC) curve for eyes with sight threatening diabetic retinopathy (proliferative diabetic retinopathy and/or diabetic macular edema) as diagnosed by RETevalscreening and confirmed by digital fundus photography.X axis shows fraction of 1-specificity while Y axis shows fraction of sensitivity. The area under the red line depicts Area Under Curve (AUC
Figure 4. The Receiver Operating Characteristic (ROC) for eyes with non-proliferative and proliferative diabetic retinopathy as diagnosed by RETevalscreening and confirmed by digital fundus photography. X axis shows fraction of 1-specificity while Y axis shows fraction of sensitivity. The area under the red line depicts Area Under Curve (AUC).
Figure 5. The Receiver Operating Characteristic (ROC) for eyes with any stage of diabetic retinopathy as diagnosed by RETeval screening and confirmed by digital fundus photography. X axis shows fraction of 1-specificity while Y axis shows fraction of sensitivity. The area under the red line depicts Area Under Curve (AUC).
| Qualitative variables | Number | Percentage proportion | |
| Sex | Male
Female |
263
137 |
65.8
34.3 |
| Glycemic control | HbA1C ≤7
HbA1C >7 |
107
293 |
26.8
73.3 |
| Past screening for diabetic retinopathy | Yes
No |
279
121 |
69.8
30.3 |
| How many times DR screening | Once
Twice 3 plus |
243
16 20 |
60.8
4.0 4.9 |
| DR treated in past | Yes
No |
105
295 |
26.3
73.8 |
DR – diabetic retinopathy, HbA1C – glycosylated haemoglobin
Table 2. Status of diabetic retinopathy based on digital photography evaluation by retina specialist
| Diabetic patients (400) | Right eye | Left eye | Worse eye | ||||
| Number | % | Number | % | Number | % | 95% CI | |
| Mild NPDR | 93 | 23.3 | 90 | 22.3 | 110 | 27.5 | 23.1 – 31.9 |
| Moderate NPDR | 0 | 0.0 | 0.0 | 0.0 | 0 | 0.0 | – |
| Severe NPDR | 34 | 8.5 | 34 | 8.5 | 40 | 10 | 7.1 – 12.9 |
| PDR | 34 | 8.5 | 40 | 10.0 | 45 | 11.3 | 8.2 – 14.4 |
| Diabetic macular edema | 55 | 13.8 | 52 | 13.0 | 54 | 13.5 | 10.2 – 16.8 |
| Sight threatening diabetic retinopathy (PDR, DME) | 80 | 20.0 | 82 | 20.5 | 108 | 27.0 | 22.6 -31.4 |
| DR of any stage | 161 | 40.3 | 163 | 40.8 | 195 | 48.8 | 43.9- 53.7 |
| No diabetic retinopathy | 169 | 42.3 | 168 | 42.0 | 149 | 37.3 | 32.6 – 42.4 |
| Not possible | 70 | 17.6 | 69 | 17.1 | 56 | 14.1 | 10.7 – 17.5
|
NPDR – non-proliferative diabetic retinopathy, PDR – proliferative diabetic retinopathy, DME – diabetic macular edema, DR – diabetic retinopathy
retinopathy (STDR) compared to digital fundus photographs evaluated by retina specialist.
| Digital photography | ||||
| STDR + | STDR – | |||
| RETeval | Fail | 103 | 156 | 259 |
| Pass | 5 | 33 | 38 | |
| 108 | 189 | 297 | ||
Sensitivity: 103/108*100= 95.4%
Specificity: 33/189*100 = 17.5%
Positive predictive value: 103/259*100 = 39.8%
Negative predictive value: 5/38*100 = 13.2%
False positives: 156/259*100 = 60.2%
False negatives: 5/108*100 = 4.6%
Table.4 Validity of RETeval in screening for diabetic retinopathy (DR) of all grades compared to digital fundus photographs evaluated by retina specialist
| Digital photography | ||||
| DR + | DR – | |||
| RETEval | Fail | 184 | 119 | 303 |
| Pass | 11 | 29 | 40 | |
| 195 | 148 | 343 | ||
Sensitivity: 184/195*100= 94.4%
Specificity: 29/148*100 = 19.6%
Positive predictive value: 184/303*100 = 60.7%
Negative predictive value: 11/40*100 = 27.5%
False positives: 119/303*100 = 39.3%
False negatives: 11/195*100 = 5.6%