Virtually Serving the Underserved: Resident Perceptions of Telemedicine Use While Training During Coronavirus Disease 2019
Introduction
The coronavirus disease 2019 (COVID-19) pandemic caused health care workers to face increased workloads, and new measures were implemented to reduce the spread of the virus and decrease the burden on the health care system. Increased use of telemedicine (TM) was one of the changes adopted by hospitals during the pandemic.1–4 Resident physicians are an integral part of the health care system and residency training involves conducting outpatient visits. This was also transformed, with increased use of TM in primary care clinics, during the pandemic.
The use of TM in the outpatient setting by medical specialties such as psychiatry, dermatology, and family medicine (FM) were studied before COVID-19.5–7 Before this study, there were few published data on the effects of COVID-19 on the delivery of resident-driven health care in the outpatient setting. Resident physicians in different specialties were also known to report burnout before the COVID-19 pandemic.8–11 With shifts in work-life balance and workplace conditions during this new global pandemic, these factors that are known determinants of physician well-being were affected.12
At the Morehouse School of Medicine (MSM), each residency class comprised at least 80% minority physicians. The mission of the MSM is to “…address primary health care through programs in education, research, and service; With emphasis on people of color and the underserved urban and rural populations in Georgia, the nation, and the world.” As such, we piloted a two-part study to: (1) investigate perceptions among residents, within primary care spaces, of transitioning to more TM during COVID-19 and (2) objectively measure burnout among internal medicine (IM) and FM residents who conducted outpatient visits during COVID-19. We hypothesized that residents will find TM to be a secure and effective alternative to face-to-face encounters, use TM for screening and providing patient education on COVID-19, and favor transitioning more visits to TM after the pandemic. We also hypothesized that burnout scores will be decreased after increased use of TM during COVID-19.
Materials and Methods
Study Design and Subjects
We conducted a retrospective observational cohort study facilitated by a survey of residents of all postgraduate years (PGYs) of training for two primary care medical specialties, IM and FM, at the MSM. This survey tool was sent electronically to a total of 78 residents, in the IM and FM programs, who completed outpatient visits during the onset of the COVID-19 pandemic from March to May 2020. Trainees were contacted via e-mail seeking anonymous, voluntary responses to an online survey without a participation incentive. The survey was conducted during a 2-week period from May 5 to May 19, 2020. By May 2020 of the COVID-19 pandemic, there were 29,839 cases of COVID-19 infection in the state of Georgia and 23,000 TM visits had been completed at the Grady Memorial Hospital, which is a primary site for MSM outpatient clinics.
Survey Instrument
The questionnaire was designed by the authors of the study and divided into three sections. Section 1 consisted of 20 multiple-choice questions with eight questions using a 5-point Likert scale for resident satisfaction with TM, screening capability for COVID-19, effectiveness to convey patient education on COVID-19, impact on patient interactions, perception of safety in health care delivery, time management, and utility of TM during COVID-19 (Appendix A1).
Sections 2 and 3 used the Abbreviated Maslach Burnout Inventory (aMBI), a validated tool,13 which includes 9 items to score overall burnout (OB). These 9 items have 3 subscales with 3 questions in each subscale assessing: emotional exhaustion (EE), emotional depletion because of job and work-related stress; depersonalization (DP), indifferent response to the recipient service; and personal accomplishment (PA), degree of achievement and satisfaction with work. Each item in the aMBI survey is a 7-point Likert scale that ranges from never (0) to every day (6). Summation of the subscale questions will have range 0–18 for each subscale. The higher score the greater burnout was for EE and DP while the lower the greater burnout for PA. For EE and DP, 0–9 was categorized as “no to low burnout” and 10–18 was regarded as “moderate to severe burnout.” For PA, 0–9 was categorized as “moderate to severe burnout” and 10–18 as “no to low burnout.” Residents were required to reflect on burnout before increased use of TM for outpatient visits during COVID-19 and compare their burnout after increased TM. The 22-item Maslach Burnout Inventory is the gold standard for burnout measurement but it has been shortened in prior studies of physicians when its length and expense make alternative tools more feasible.10
Data Analysis
Data Analysis for Section 1 was carried out with Microsoft Excel and STATA software. For Sections 2 and 3, descriptive statistics were performed using median with minimum and maximum for aMBI subscale scores and frequency with percentage for categorical variables. The Wilcoxon signed-rank tests were performed to compare the score changes from before to after, and the two independent samples t-test were performed to compare the score changes between IM and FM. All the data analyses were performed using SAS 9.4. Values of p < 0.05 were considered statistically significant.
This study was approved by the MSM Biomedical Institutional Review Board.
Results
A total of 62 of 78 eligible residents (79.5%) from the IM and FM residency programs completed the survey. Forty-eight of 60 IM residents (80%) and 14 of 18 FM residents (77.8%) responded. Most of the residents who responded were women (61%), which matched the overall demographics of the IM and FM programs. Among the residents who responded, 39% were PGY1, 35% were PGY2, and 26% were PGY3. After excluding residents who completed the survey but indicated they did not use TM, 57 responses were eligible for data analysis.
Fourteen percent of residents, mainly PGY2 and PGY3, were exposed to outpatient settings that involved the use of TM before the COVID-19 pandemic. Ninety-seven percent of residents who responded to the survey indicated that they completed TM through telephone calls and 37% had also used video conferencing during COVID-19. There was a decrease in the average perceived length of time of visits with the use of TM as opposed to traditional face-to-face encounters by an average length of 10–20 min (Fig. 1). Seventy-one percent of residents strongly agreed or agreed that TM improved time management during the COVID-19 pandemic, with 25% remaining neutral, 5% disagreeing, and no resident strongly disagreeing (Fig. 2).
Fig. 1. Average perceived time comparison for face–to-face and telemedicine outpatient visits.
Fig. 2. Resident feedback on telemedicine use (There were no nonstandard abbreviations used in the figures for this article). COVID-19, coronavirus disease 2019.
Most of residents (60%) strongly agreed or agreed that the introduction of TM reduced anxiety/nervousness about outpatient encounters at the beginning of the COVID-19 pandemic, 26% remained neutral, and 14% disagreed or strongly disagreed. TM was perceived as a secure option for continued outpatient encounters during COVID-19 pandemic by 91% of residents.
TM was used for outpatient follow-up of established clinic patients with chronic medical illnesses. Other uses of TM in clinic during COVID-19 included the following: to screen patients for early symptoms of COVID-19 by 79% of residents, to provide patient education on COVID-19 by 93% of residents, and to refer patients to specialty/consult services that are needed during COVID-19 by 82% of residents. Eighty-nine percent of residents strongly agreed or agreed that they were able to adequately address patient concerns during TM visits, whereas the other 11% were neutral with no one disagreeing. Sixty-five percent of residents strongly agreed or agreed that more visits in the outpatient clinical setting should transition from face-to-face encounters to virtual encounters after the COVID-19 pandemic, whereas 25% were neutral, and 11% disagreed (Fig. 2).
THE aMBI
The aMBI analysis, as shown in Table 1, revealed that residents had a score of 8 for EE before the implementation of TM owing to the onset of COVID-19, and a score of 7 post-TM (p = 0.017). DP, another subscale of the aMBI, had scores of 8 pre-TM and 7 post-TM (p = 0.0001). Although PA showed an unchanged score of 6 pre-TM and post-TM, there was no statistical significance (p = 0.755). OB measured by the addition of EE and DP demonstrated a score of 16 pre-TM and 14 post-TM (p = 0.0003).
| AMBI SUBSCALE | SCORE BEFORE MEDIAN (MIN, MAX) | SCORE AFTER MEDIAN (MIN, MAX) | MEDIAN DIFFERENCE (MIN, MAX) | p |
|---|---|---|---|---|
| EE | 8 (2, 16) | 7 (3, 13) | 1.00 (−5, 6) | 0.0173 |
| DP | 8 (4, 17) | 7 (0, 15) | 1.25 (−4, 12) | <0.0001 |
| PA | 6 (0, 14) | 6 (0, 12) | 0.26 (−5, 13) | 0.7548 |
| EE + DP = OB | 16 (10, 33) | 14 (6, 25) | 1.95 (−5, 18) | 0.0003 |
When the absolute distribution of each subscale among residents was analyzed, as given in Table 2 for DP, 33% of residents had moderate to high burnout pre-TM with a decrease to 18% of residents post-TM (p = 0.004). EE and PA were unchanged pre-TM and post-TM (p = 1.000). OB, using EE and DP, showed moderate to high burnout improving from 32% pre-TM to 19% post-TM (p = 0.0156).
| BEFORE | AFTER | ||||
|---|---|---|---|---|---|
| NO TO LOW BURNOUT, N (%) | MODERATE TO HIGH BURNOUT, N (%) | NO TO LOW BURNOUT, N (%) | MODERATE TO HIGH BURNOUT, N (%) | p | |
| EE | 41 (72) | 16 (28) | 41 (72) | 16 (28) | 1.0000 |
| DP | 38 (67) | 19 (33) | 47 (82) | 10 (18) | 0.0039 |
| PA | 5 (9) | 52 (91) | 4 (7) | 53 (93) | 1.0000 |
| EE + DP = OB | 39 (68) | 18 (32) | 46 (81) | 11 (19) | 0.0156 |
Discussion
This resident-oriented study gauged perceptions and effects of TM use within primary care clinics during the COVID-19 pandemic. Residents in the IM and FM programs had an overall positive response to transitioning to use of TM in the outpatient setting. Use of telephone calls was the most common means of completing virtual visits, whereas a minority also had experience with the secure and encrypted video-conferencing system platform called PROCLE Healthworks. Although TM was used at MSM to address any outpatient chronic medical concerns, TM has also been specifically recommended for the detection and management of COVID-19.2 Most MSM residents reported use of virtual platforms to screen patients for early symptoms/signs of COVID-19 and provide patient education on the illness. Eighty-nine percent of residents strongly agreed or agreed that they were able to address medical concerns adequately with TM.
These findings are in keeping with studies carried out during previous pandemics such as Ebola and Influenza A virus subtype H1N1.2–4 They may likely be of benefit for future pandemics. Before COVID-19, a 2015 study showed that only 57% of FM residents used telehealth services,14 and another study of 2016 data showed that 38% of community centers offered telehealth services.15 With increased use of TM at our institution, it is feasible for some patient evaluations to remain through TM after the COVID-19 pandemic, depending on the need for laboratory work or urgent medical management. The majority of residents (91%) agreed or strongly agreed that TM was a safe alternative to face-to-face interaction with patients. The survey did not specifically ask detailed questions about safety through TM.
The decreased length of visits reported with TM may have resulted from reduced clinic check-in process, the ability to contact patients earlier, and no physical examination.
This has implications on reimbursement as charges are proportional to length of TM visits and less revenue is generated with shorter visits. IM and FM residents received training on TM through online workshops led by the MSM directors of TM and outpatient clinics. The training included TM techniques, workflow, computer skills, etiquette, scheduling, billing, and follow-up services. Workshops were provided for precepting attendings and ancillary outpatient clinic staff. There were ongoing updates through weekly didactics and mass electronic communication. By May 2020 of the COVID-19 pandemic, 23,000 TM visits were completed at Grady Memorial Hospital, a primary outpatient location for the MSM outpatient clinics.
In addition, the questionnaire in this study was used to measure any change in burnout among residents with increased use of TM during COVID-19. Burnout rates among primary care physicians were studied before COVID-19. In one study, 44% of physicians reported feeling burned out, 11% were colloquially depressed, and 4% were clinically depressed.8 Resident physicians undergoing their training within IM and FM programs have also reported burnout.9,10 Residents completed the aMBI based on how they felt before and after the introduction/increased use of TM for outpatient visits during COVID-19. This showed a statistically significant reduction in EE among residents with the introduction of TM. DP also showed a statistically significant improvement in burnout measurement. Although PA was unchanged with use of TM, there was no statistical significance. Therefore, by the addition of EE and DP, TM reduced perceived OB among residents. Burnout has been associated with a reduction in the quality of patient care and an increase in medical errors.8,11 By reducing burnout, not only will residents benefit but resident-driven patient care will also improve.
Limitations of this study include variation in comfort with continuity clinic visits based on PGY level, and differences in burnout based on residency program and PGY level. Sampling bias was a limitation as the online survey may have been completed by residents with a higher level of computer expertise and experience with electronic surveys. There was possible recall bias, as there were no data collected before increased use of TM and COVID-19. As a result of coding, policy, and reimbursement regulations that were still being developed during the pandemic, subjective resident estimates for virtual visit times was the primary means of gathering data. Residents who were on rotations that did not include conduction of outpatient visits were not able to participate in this study. There was also variation in how many outpatient visits each resident completed. Our questionnaire did not account for how many visits were completed through video versus telephone. Although the aMBI in Sections 2 and 3 is a validated tool,13 Section 1 was not tested for validity evidence. This survey did not include patient perspectives on health care delivery. This study was completed over only a 2-week period and included only two primary care specialties. Further studies can be carried out over a longer period of time and to include other specialties and thereby increase external validity of findings. Burnout among other outpatient clinicians such as attending physicians and nurse practitioners can also be studied.
Conclusions
Most residents perceived TM as a secure alternative to face-to-face encounters for follow-up of chronic medical illnesses, and it was perceived to improve time management and decrease nervousness/anxiety during COVID-19. Most residents believed patient concerns were adequately addressed with TM and it was also used to provide education and screen for symptoms/signs of COVID-19. There was an overall decrease in burnout among residents with increased use of TM during the COVID-19 pandemic.
Authors’ Contributions
K.M.: Conception and design, acquisition of data, analysis and interpretation of data, drafting and revision of article, final approval. A.P.: Conception and design, acquisition of data, drafting and revision of article, final approval. M.M.: Conception and design, acquisition of data, drafting and revision of article, final approval. T.A.: Conception and design, acquisition of data, drafting and revision of article, final approval. J.I.: Analysis and interpretation of data, final approval. T.S.: Conception and design, acquisition of data, analysis and interpretation of data, drafting and revision of article, final approval. P.S.: Acquisition of data, drafting and revision of article, final approval. T.A.: Acquisition of data, analysis, and interpretation of data, final approval. F.Y.: Analysis and interpretation of data, final approval. E.C.: Acquisition of data, drafting and revision of article, final approval. J.V.: Drafting and revision of article, final approval.
Acknowledgments
The authors thank Robert Mayberry from the Department of Community Health and Preventative Medicine for input on data analysis.
Disclosure Statement
The authors declare they have no competing interests.
Funding Information
The authors report no external funding source for this study.
REFERENCES
- 1. . COVID-19 and telemedicine: Immediate action required for maintaining healthcare providers well-being. J Clin Virol 2020;126:104345. Crossref, Medline, Google Scholar
- 2. . The use of digital health in the detection and management of COVID-19. Int J Environ Res Public Health 2020;17:2906. Crossref, Google Scholar
- 3. Precision global health–the case of ebola: A scoping review. J Glob Health 2019;9:010404. Crossref, Medline, Google Scholar
- 4. . SARS and population health technology. J Med Internet Res 2003;5:e14. Crossref, Medline, Google Scholar
- 5. . Usefulness of telepsychiatry: A critical evaluation of videoconferencing-based approaches. World J Psychiatry 2015;5:286–304. Crossref, Medline, Google Scholar
- 6. Multicentre randomised control trial comparing real time teledermatology with conventional outpatient dermatological care: Societal cost-benefit analysis. BMJ 2000;320:1252–1256. Crossref, Medline, Google Scholar
- 7. . Family physicians report considerable interest in, but limited use of, telehealth services. J Am Board Fam Med 2017;30:320–330. Crossref, Medline, Google Scholar
- 8. Rates of medication errors among depressed and burnt out residents: Prospective cohort study. BMJ 2008;336:488–491. Crossref, Medline, Google Scholar
- 9. . Association of resident fatigue and distress with perceived medical errors. JAMA 2009;302:1294–1300. Crossref, Medline, Google Scholar
- 10. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med 2012;172:1377–1385. Crossref, Medline, Google Scholar
- 11. Primary care tasks associated with provider burnout: Findings from a Veterans Health Administration survey. J Gen Intern Med 2018;33:50–56. Crossref, Medline, Google Scholar
- 12. Physician, practice, and patient characteristics related to primary care physician physical and mental health: Results from the Physician Worklife Study. Health Serv Res 2002;37:121–134. Medline, Google Scholar
- 13. The Abbreviated Maslach Burnout Inventory can overestimate burnout: A study of anesthesiology residents. J Clin Med 2020;9:61. Crossref, Google Scholar
- 14. . Over half of family medicine residency program directors report use of telehealth services. Telemed J E Health 2019;25:933–939. Link, Google Scholar
- 15. . Telehealth in health centers: Key adoption factors, barriers, and opportunities. Health Aff 2018;37:1967–1974. Crossref, Medline, Google Scholar
Appendix
Appendix A1. Appendix Questionnaire
| 1. |
Select Your Residency Program: |
||||
|
a. Family Medicine |
|||||
|
b. Internal Medicine |
|||||
| 2. |
Gender: |
||||
|
a. Female |
|||||
|
b. Male |
|||||
| 3. |
PGY level of training: |
||||
|
a. 1 |
|||||
|
b. 2 |
|||||
|
c. 3 |
|||||
|
d. 4 |
|||||
| 4. |
I used telemedicine in the outpatient clinical setting BEFORE the COVID-19 pandemic: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
| 5. |
I have used telephone calls to complete virtual encounters with patients: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
| 6. |
I have used video conferencing to complete virtual encounters with patients: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
| 7. |
Average length of time for face-to-face outpatient encounters: |
||||
|
a. <10 min |
|||||
|
b. 10–20 min |
|||||
|
c. 20–30 min |
|||||
|
d. 30–40 min |
|||||
|
e. 40–50 min |
|||||
|
f. 50–60 min |
|||||
|
g. >60 min |
|||||
| 8. |
Average length of time for telemedicine outpatient encounters: |
||||
|
a. <10 min |
|||||
|
b. 10–20 min |
|||||
|
c. 20–30 min |
|||||
|
d. 30–40 min |
|||||
|
e. 40–50 min |
|||||
|
f. 50–60 min |
|||||
|
g. >60 min |
|||||
| 9. |
Telemedicine in clinics optimizes my time to attending to inpatients who may have COVID-19 or another critically ill condition: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 10. |
I felt anxious/nervous about my general clinical responsibilities as a resident at the beginning of the COVID-19 pandemic: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 11. |
I felt anxious/nervous about my outpatient encounters at the beginning of the COVID-19 pandemic: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 12. |
The introduction/increased use of telemedicine reduced my anxiety/nervousness about outpatient encounters at the beginning of the COVID-19 pandemic: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 13. |
I think telemedicine offers a secure option for continued outpatient encounters during COVID-19 pandemic: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 14. |
I have used telemedicine, while in quarantine at home after exposure to COVID-19, to continue to deliver patient care: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
|
c. Does not apply to me |
|||||
| 15. |
I have used telemedicine to screen patients for early symptoms/signs of COVID-19: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
| 16. |
I have used telemedicine to provide patient education on COVID-19: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
| 17. |
I am able to adequately address patient concerns during telemedicine visits: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 18. |
I have used telemedicine to refer patients to specialty/consult services that are needed during COVID-19: |
||||
|
a. Yes |
|||||
|
b. No |
|||||
| 19. |
Telemedicine has negatively impacted the quality of my patient care during COVID-19: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
| 20. |
I believe visits in the outpatient clinical setting should transition from face-to-face encounters to virtual encounters after the COVID-19 pandemic: |
||||
|
a. Strongly disagree |
|||||
|
b. Disagree |
|||||
|
c. Indifferent |
|||||
|
d. Agree |
|||||
|
e. Strongly agree |
|||||
|
Comment below on reason for selected answers (optional) |
|||||
|
Abbreviated Maslach Burnout Inventory |
|||||
| 21. |
With no/limited use of telemedicine for outpatient encounters BEFORE COVID-19: |
||||
| EVERY DAY | A FEW TIMES A WEEK | ONCE A WEEK | A FEW TIMES A MONTH | ONCE A MONTH OR LESS | A FEW TIMES A YEAR | NEVER | |
|---|---|---|---|---|---|---|---|
| I deal very effectively with the problems of my patients | |||||||
| I feel I treat some patients as if they were impersonal objects | |||||||
| I feel emotionally drained from my work | |||||||
| I feel fatigued when I get up in the morning and have to face another day on the job | |||||||
| I’ve become more callous toward people since I took this job | |||||||
| I feel I’m positively influencing other people’s lives through my work | |||||||
| Working with people all day is really a strain for me | |||||||
| I don’t really care what happens to some patients | |||||||
| I feel exhilarated after working closely with my patients |
|
Abbreviated Maslach Burnout Inventory |
|||||
| 22. |
With increased use of telemedicine for outpatient encounters for SINCE COVID-19: |
||||
| EVERY DAY | A FEW TIMES A WEEK | ONCE A WEEK | A FEW TIMES A MONTH | ONCE A MONTH OR LESS | A FEW TIMES A YEAR | NEVER | |
|---|---|---|---|---|---|---|---|
| I deal very effectively with the problems of my patients | |||||||
| I feel I treat some patients as if they were impersonal objects | |||||||
| I feel emotionally drained from my work | |||||||
| I feel fatigued when I get up in the morning and have to face another day on the job | |||||||
| I’ve become more callous toward people since I took this job | |||||||
| I feel I’m positively influencing other people’s lives through my work | |||||||
| Working with people all day is really a strain for me | |||||||
| I don’t really care what happens to some patients | |||||||
| I feel exhilarated after working closely with my patients |