Evaluation of a Rapid Implementation of Telemedicine for Delivery of Obstetric Care During the COVID-19 Pandemic

Introduction

The 2019 novel coronavirus (COVID-19), originating in Wuhan, China, subsequently became a global pandemic infecting over 250 million people and resulting in over 5 million deaths to date.¹ In the United States, >50 million people have tested positive for COVID-19 and over 700,000 deaths have occurred.² In the initial stages of the COVID-19 pandemic, lockdowns, quarantines, and social distancing measures were implemented because of the high transmissibility of the COVID-19 virus, shortage of personal protective equipment, and the absence of curative treatments or a vaccine.

As a result, these measures impeded the provision and participation of traditional in-person health care appointments. For obstetric patients, the need to limit infection exposure, reduced access to childcare because of school closures, and restrictions to working from home were barriers likely to prevent access to in-person obstetric care. Likewise, in response to the COVID-19 pandemic, preventive social measures were likely to impact health-related policies and practices thereby limiting the delivery of comprehensive obstetric care services.³ To support the continuity of health care services during the COVID-19 pandemic while adhering to pandemic restrictions, there was an increase in the adoption and utilization of telemedicine for the delivery of health care services.

Telemedicine involves two-way audio and visual electronic communication between patients and providers in real-time at different remote locations.⁴ The American Hospital Association reported that in 2017, 76% of U.S. hospitals connected patients remotely with health care providers using video or other technology.⁵ Research has also shown that before the COVID-19 pandemic, Radiology specialists had the highest utilization rates of telemedicine (39.5%), while Obstetrician-Gynecologist specialists had among the lowest utilization rates of telemedicine (9.3%) in their practices.⁶

Telemedicine has been shown to be advantageous for both patients and health care providers, due to its ease of accessibility to attend office visits from different geographical locations, reduced expenses related to travel costs and childcare costs, increased time efficiency for consultations, and limited exposure to infectious diseases.⁷

In addition, obstetric telemedicine has been found to have similar safety and effectiveness to in-person care,⁸ and no differences have been found in the rates of health outcomes such as gestational age at birth, preterm births, cesareans, birthweight, and neonatal intensive care admissions when comparing obstetric telemedicine visits to in-person visits.⁹

Although most studies have assessed the implementation of telemedicine in various clinical settings without the guidance of theory,^10–16 we aimed to assess the rapid implementation, feasibility, and satisfaction of obstetric telemedicine during the COVID-19 pandemic using the Consolidated Framework in Implementation Research (CFIR) evaluation framework.¹⁷

Subjects and Methods

OVERVIEW

This qualitative evaluation was developed using CFIR, which provides constructs that have been associated with effective implementation (Table 1).¹⁸ Key CFIR constructs guiding this evaluation included Intervention Characteristics (Evidence Strength and Quality, Relative Advantage, Adaptability, and Complexity), Outer Setting (Patient Needs and Resources), Inner Setting (Implementation Climate and Readiness for Implementation), Characteristics of Individuals (Knowledge and Beliefs about the Intervention), and Processes (Engaging Intervention Participants) related to the implementation of obstetric telemedicine.

DATA COLLECTION

We recruited obstetric providers delivering obstetric care visits via telemedicine at the University of South Florida clinic from May to November 2020. Purposive sampling was used to recruit obstetric providers via email who were provided with a link to complete an anonymous online survey. Eligibility criteria included health care providers that currently provided obstetric care at University of South Florida clinic. Nineteen participants completed the online survey out of 31 total prenatal care providers in the practice (Table 2).

The anonymous online Qualtrics survey included a demographic questionnaire on providers’ age, race, and years in practice postresidency or fellowship training, and open-ended questions guided by key CFIR constructs.¹⁹ We selected eight key CFIR constructs most relevant to implementation of obstetric telemedicine. Some open-ended survey questions included: “What advantages do telemedicine visits have compared to existing programs/in-patient visits?” (Intervention Characteristics); “What barriers do you believe obstetric patients face in participating in telemedicine visits?” (Outer Setting); “What was the general level of receptivity in your clinic to implementing telemedicine visits?” (Inner Setting); “What steps were taken to encourage patients to commit to using telemedicine visits?” (Process) (full survey available on request).

Following the completion of surveys, we invited providers meeting the inclusion criteria to participate in semistructured interviews to ascertain more in-depth feedback regarding their experience in implementation of obstetric telemedicine. A total of 13 providers, some of whom previously completed the anonymous online survey, participated in the in-depth interviews. These interviews consisted of questions identical to those in the online survey, with opportunity to expand upon answers. Interviews lasted ∼20 to 30 min and were audio recorded and transcribed for data analysis. To preserve confidentiality, each transcript was assigned a pseudonym. This study was reviewed and determined to be exempt as program evaluation by the University of South Florida Institutional Review Board.

DATA ANALYSIS

We developed an initial codebook based on a priori structural codes guided by CFIR. This codebook was revised for clarity and the relevance of codes and definitions. Two evaluation team members independently coded four transcripts to consensus (C.R. and K.F.), and a kappa statistic of 0.8 was calculated, indicating good agreement between coders.²⁰ Any coding discrepancies were identified and discussed to achieve agreement on the final codebook. One coder (C.R.) coded the remaining surveys and transcripts. A thematic analysis approach was used to analyze the data²¹ and all coding and data analysis was conducted using MAXQDA 2020 software.²² Recruitment of providers was ended once it was determined that data saturation was achieved; participant comments were repeated from previous surveys/interviews and no new information was being added.

In addition, trustworthiness was enhanced by conducting peer debriefing (credibility), having two evaluation team members independently code at least 10% of the interviews with a good kappa (reliability), using provider quotes to represent themes (confirmability), and by having audit trails throughout the process.^23,24

Results

The evaluation results from survey and interview participants (Table 2) are summarized below along with representative quotes found in Table 3 (quote ID noted in parentheses). These findings are aligned with CFIR constructs that help to inform the pros and cons of obstetric telemedicine implementation from the viewpoints of a diverse range of providers who are newly conducting this practice. Doximity and Microsoft Teams programs on HIPAA-secure computer devices were used to conduct telemedicine consultations.

PROVIDER DESCRIPTION

The participants in this study were predominantly white (74%), with black (11%), Hispanic (11%), and Asian (5%) providers represented as well (Table 2). Over a third (37%) had been in practice for up to 5 years, while over 58% had been in practice for >15 years.

PROVIDER PERCEPTIONS OF IMPLEMENTATION

Before this implementation of obstetric telemedicine, almost all providers had never used telemedicine in practice before. With the rapid implementation of obstetric telemedicine early in the COVID-19 pandemic, obstetric providers shared their perceptions.

QUALITY

The quality of telemedicine visits referred to providers’ perception of the safety (avoiding risk or harm) and efficacy of telemedicine visits to have desired outcomes. Most providers did not have any concerns about the safety and efficacy of telemedicine visits. However, a few providers (11 years or more in practice) had concerns about solely using telemedicine for new patients with unrecognized urgent conditions or for patients who required a physical examination or ultrasound (IC1).

ADAPTABILITY

The adaptability of telemedicine visits entailed the degree to which changes could be made to telemedicine visits for it to work effectively in a clinic. Such changes included the use of multiple platforms (e.g., Microsoft Teams, Doximity dialer, etc.) to conduct telemedicine visits efficiently, the flexibility of both scheduling patient appointments and allotting extra time to transition between scheduled in-person and telemedicine appointments, or simply for troubleshooting technical issues if necessary. Likewise, the ability to incorporate support staff during telemedicine visits to improve workflow made telemedicine work effectively (IC2).

RELATIVE ADVANTAGE

The relative advantage of telemedicine visits referred to providers’ perception of the advantages of implementing obstetric telemedicine compared to in-person obstetric visits. Generally, providers regarded telemedicine as comparable to in-person obstetric visits. Convenience, comfort, and decreased exposure to COVID-19 were the most common themes associated with the relative advantage of obstetric telemedicine.

Convenience was described by providers as less travel commute and parking issues, less visit wait times, reduced need for childcare, fewer missed or late appointments, more flexibility in scheduling appointments, and longer visits for counseling. In addition, providers expressed added benefits of telemedicine for both patients and providers to include the convenience of less distractions and a decreased risk of exposure to COVID-19 infection (IC3). Providers also discussed the personal connections they were able to have with patients because of how comfortable patients were in their home environment during telemedicine visits (IC4).

Overall, providers described telemedicine as having several advantages over traditional in-person visits. Barriers such as limitations in performing physical examinations, laboratories, or ultrasounds when deemed necessary were addressed by scheduling the patient for an in-person visit.

COMPLEXITY

The complexity of telemedicine visits was defined as the perceived difficulty of its implementation. Most providers described telemedicine visits as simple or not complicated to implement. However, a few providers in practice for 16 years or longer found telemedicine to be slightly more complex in terms of the multiple steps involved in the process of conducting a telemedicine visit (IC5).

PATIENT NEEDS AND RESOURCES

This construct encompassed the extent to which patient needs, and barriers and facilitators to meet those needs, were known and prioritized by the clinic. In the context of the COVID-19 pandemic, which restricted access to health care services, telemedicine offered an alternative option to access obstetric care.

Themes encompassing this construct—patients’ need for convenience, comfort, decreased exposure to COVID-19 infection, and physical examination, were similar to those under the relative advantage construct. For example, one provider discussed how the convenience of obstetric telemedicine visits and the flexibility to include family members for support at these visits was helpful to patients (OS1). Many providers mentioned the need expressed by patients to hear the fetus’ heartbeat during visits. However, this was only possible if the patient had an at-home fetal heart doppler machine.

Barriers to meeting these needs included a lack of patient access to at-home monitoring devices such as BP machines, weight scales, and fetal heart doppler machines to measure vital signs; health insurance policies not covering the purchase of these at-home monitoring devices; the impracticality of having a physical examination, laboratory work or ultrasound done if required (OS2); and access to technology such as reliable internet service, which was required to participate in telemedicine visits (OS3).

Some providers mentioned patient privacy and distractions as barriers to patient comfort and a more efficient telemedicine visit (OS4). Overall, providers felt that telemedicine as an alternative option to in-person obstetric visits met most of these patients’ needs. This was particularly for low-risk obstetric patients not requiring a physical examination and for patients with certain hearing or mobility limitations (OS5).

IMPLEMENTATION CLIMATE

Implementation climate referred to the shared receptivity of providers to telemedicine visits within their clinics. Overall, providers were receptive to the implementation of telemedicine visits. One provider reported that although some providers were initially nervous about the implementation of telemedicine, it was, however, well received (IS1).

READINESS FOR IMPLEMENTATION

The tangible and immediate indicators of overall commitment to the decision to implement telemedicine visits are described by the readiness for implementation construct. Regarding the question “Did you feel the training prepared you to carry out telemedicine visits?,” all providers acknowledged feeling prepared to implement telemedicine visits. We further investigated two of the three subconstructs under the main construct readiness for implementation, that is, (1) available resources and (2) access to knowledge and information.

AVAILABLE RESOURCES

This subconstruct included resources devoted for the implementation and ongoing operation of telemedicine visits. Generally, themes associated with resources available for implementation as reported by providers included technology access and support, staff support, time availability, and access to at-home monitoring devices (e.g., BP machine and so on). For some providers, resources for telemedicine implementation were considered adequate, while for others the opposite held true. For example, access to telemedicine platforms such as Microsoft Teams and Doximity and the availability of staff to troubleshoot both technical issues and assist with the telemedicine visit process were considered ample (IS2). However, some providers did not have adequate staff support (IS3).

Likewise, provision of time allotted for telemedicine visits was sufficient and appropriate for some, but not all providers (IS4). Finally, some patients had access to BP machines, weight scales, and fetal heart dopplers provided to them by the clinic, while other patients did not have access because they were not covered by insurance and were too expensive (IS5). This created a barrier for patients’ desire to hear the baby’s heartbeat and for providers to complete comprehensive telemedicine visits.

ACCESS TO KNOWLEDGE AND INFORMATION

This construct included access to available information and knowledge associated with obstetric telemedicine and its use. Providers reported generally gaining knowledge and access to information about telemedicine visits through in-person and online trainings. In particular, providers with fewer years in practice revealed that the process of familiarizing oneself with telemedicine was easier for younger or more tech savvy colleagues (IS6). One provider sought obstetric telemedicine information provided by their clinic and compared it to information provided at other larger institutions (IS7).

Although most providers gained knowledge and information from trainings offered, some indicated that there were no need for continued trainings, while others indicated certain aspects they considered missing or needing additional trainings. These included information and trainings related to telemedicine billing and reimbursements (IS8), improvements in provider–patient interaction (IS9), and troubleshooting technological issues (IS10).

KNOWLEDGE AND BELIEFS ABOUT THE INTERVENTION

The construct of knowledge and beliefs about obstetric telemedicine referred to providers’ attitudes and values placed toward telemedicine visits and its implementation in their clinics. Overall, all providers had positive reactions to implementation as it related to their knowledge and beliefs about telemedicine. Providers mostly considered telemedicine a usefulness alternative to deliver obstetric care especially for low-risk obstetric patients (IN1). In addition, some providers believed the implementation of telemedicine enabled obstetric providers to continue employment during the COVID-19 pandemic, and despite few providers believing that its implementation was rushed, some providers expressed the desire for telemedicine to continue beyond the pandemic (IN2).

ENGAGING

The question “What steps were taken to encourage patients to commit to using telemedicine visits?” referred to the construct of engaging. Mostly, providers reported that patients were encouraged to commit to participating in telemedicine visits by educating them about the simplicity and advantages of telemedicine (similar to those mentioned under the relative advantage construct), explaining to them what the process of a typical telemedicine visit entailed, as well as reassuring them of adequate patient care (Pr1). One provider reported that because of the COVID-19 pandemic, patients were not given an option to opt-out of telemedicine visits. Nevertheless, it has now become part of routine care in the clinic (Pr2).

Discussion

The implementation of the obstetric telemedicine care model in our clinical setting was deemed favorable, safe, and an alternative option for patients during the COVID-19 pandemic. The majority of providers found this model of care to be easy to use and implement even though they did not have past exposure to telemedicine. Barriers to telemedicine noted included lack of home monitoring devices for blood pressure and fetal heart tones, knowledge of billing, and concerns regarding patient privacy. These advantages and barriers are similar to those reported in other studies analyzing rapid telemedicine implementation in the post-COVID era.^10–16

It is encouraging that most of the barriers identified by providers can be addressed. Furthermore, the pros noted in this evaluation suggest that it may be advantageous to extend telemedicine beyond the pandemic to improve the efficiency and access of health care services provision in the future. The American College of Obstetricians and Gynecologists has provided guidance to providers in the form of a telemedicine implementation guide²⁵ and guide for telemedicine implementation during the pandemic.²⁶

STRENGTHS AND LIMITATIONS

One of the major strengths of this study was the ability to quickly evaluate a rapid implementation of telemedicine during a pandemic. We were able to capture this feedback during the rollout of a new model of care and in a prospective manner. Our results compare favorably with the current research, which shows that telemedicine was well received during the COVID pandemic.^10–16 In addition, we were able to perform this evaluation with a robust theoretical framework. Weaknesses include that this study is single site within the context of a unique situation (rapid implementation due to the COVID-19 pandemic); therefore, the results may differ from future implementation of telemedicine where more advanced planning and training would be possible before implementation.

FUTURE RESEARCH

The research on telemedicine-enhanced prenatal care during the pandemic is a growing field, but solid randomized trials are scant in the literature.^27,28 Further evaluation is also needed to explore this care model from patient perspectives and randomized controlled trial is needed to show safety and efficacy. In addition, more research is needed on how to integrate telemedicine into care for low-risk pregnancies, patient privacy concerns, and access to internet and at-home monitoring devices.

Telemedicine shows promise for certain populations, but may not be ideal for all due to insurance reimbursement issues, need for in-personal monitoring for high-risk pregnancies, and access to high-speed internet. Additional funding is needed to provide patients with at home monitoring devices through insurance reimbursement or other funding mechanisms, especially for low-income women who cannot afford these devices due to high out-of-pocket costs.

Further guidance and standardization of a telemedicine prenatal care model for low-risk women from national professional organizations such as American College of Obstetricians and Gynecologist and the American College of Nurse-Midwives would be advantageous for both providers and patients in implementation of this model across the country. Some clinics may find that telemedicine is best used to provide specific patient education or remote monitoring.^8,26,29

Conclusions

From the provider perspective, the rapid implementation of the obstetric telemedicine care model in a single clinical site was deemed a favorable option for patients during the COVID-19 pandemic and shows promise for further study. Future research and evaluation are needed to assess patient satisfaction, safety, and compare maternal/neonatal outcomes compared to in-person only care.

Disclosure Statement

No competing financial interests exist.

Funding Information

This work was supported by University of South Florida Internal Seed Grant (Grant number IS415-11).