Society for Education and the Advancement of Research in Connected Health AbstractsSEARCH 2021 – The National Telehealth Research Symposium November 8–10, 2021 Virtual Meeting

SEARCH PRESENTATIONS

2. Utilization of a Prenatal Telehealth Simulation with Prelicensure Nursing Students

Elizabeth Riley DNP, Nicole Ward, PhD, Colton McCance DNP, Pamela deGravelles PhD

University of Arkansas for Medical Sciences

Background: The use of telehealth in an simulation can help meet the strategic goal of integrating telehealth competencies within nursing curricula (Smith, Watts, & Moss, 2018). Research is available on incorporation of telehealth simulation in nurse‐practitioner programs. However, the literature remains limited on the implementation and evaluation of telehealth simulations in prelicensure, undergraduate nursing education (Knight & Prettyman, 2020; Vaona et al., 2017). The new AACN Essentials specifically discusses the importance of teaching undergraduate students the basic concepts of telehealth systems for patient care, which falls under Domain 8, “Informatics and Healthcare Technologies” (American Association of Colleges of Nursing [AACN, 2021]). Based on this revision to the Essentials, telehealth is an important aspect to incorporate into the undergraduate nursing curriculum to aid nursing students in learning to deliver safe patient care using enhanced communication technology.

Methods: The purpose of this presentation is to describe the use of a telehealth simulation to teach prelicensure nursing students about appropriate prenatal care, patient education, and anticipatory guidance in telehealth settings. The INACSL Standards were utilized in the design of the scenarios in the simulation by instructors in the undergraduate Nursing Care of the Childbearing Family course in conjunction with the simulation certified college simulation director. Each trimester scenario had a specific protocol that was used by the simulation facilitators using a standardized participant. The students were given the learning objectives, case details regarding obstetric history, laboratory results, and current medications one week before the simulation. The telehealth simulation was evaluated using the Simulation Effectiveness Tool—Modified (SET‐M) to evaluate participant perception of the effectiveness of learning in the simulation environment. The SET‐M is a valid and reliable tool with four subscales (prebriefing, learning, confidence, and debriefing) used to evaluate the perceptions of participants regarding their learning experience during simulation scenarios (α ranging from .833‐.913) (Leighton et al., 2015).

Results: The mean participant score of the SET‐M was 2.92/3.00 and the mean score for the response prompt, “I am more confident in communicating with my patient,” was 2.95/3.00 (n = 95). Several participants gave positive reviews in the open‐response section of the survey, which generated reoccurring comments related to positive aspects of learning how to communicate with a patient in telehealth settings, debriefing and transfer of knowledge after the interaction, and building trust and rapport with patients. The main area for improvement based on open responses was the need for the standardized participant to give feedback to each group after the interaction rather than overall feedback at the end of the simulation.

Discussion: The mean scores for the SET‐M and confidence item in communication are meaningful data. Specifically, Leighton et al. (2015) discuss the alignment of measuring participant communication after simulations to student confidence. In their analysis, the item regarding communication was most strongly associated with the confidence subscale. Communication is a concept discussed in the BSN Essentials and directly relates to several domains (Domain 2: Person‐Centered Care, Domain 3: Population Health, Domain 6: Interprofessional Partnerships, and Domain 8: Informatics and Healthcare Technologies) (AACN, 2021). Based on these findings, communication, especially in telehealth settings, is vital to aiding student confidence to provide appropriate nursing care and anticipatory guidance. The simulation can be adapted to address a multitude of telehealth competencies, prenatal education, communication, and the social determinants of health by changing pertinent patient history in the scenario.

3. User security behavior among healthcare professionals in the digital age: A comparative approach

Sanjana Sundara Raj Sreenath B.S., Barbara Hewitt PhD; Sahana Sundara Raj Sreenath MS

Texas State University

Background: Given the rapid advancements in digital health and increasing reliance on technology in healthcare, security has become more important than ever As the number of data breaches in healthcare continues to rise, keeping the digital health sphere secure has become more important than ever. This interdisciplinary Health Information Technology research study investigates security behavior among healthcare professionals by drawing upon two prevalent theories in information security (IS) research: Protection Motivation Theory (PMT) and Technology Threat Avoidance Theory (TTAT).

Although there is increasing IS research using these models, very few studies comparing the two models, specifically in a healthcare context, for exploring user security behavior exist. This study, by comparing PMT and TTAT, seeks to further enrich our understanding of the factors influencing security behavior of healthcare professionals.

Methods: Thirteen hypotheses to compare the two models were developed and tested using a diverse sample of 245 surveyed healthcare professionals. Construct reliability, convergent validity, and discriminate validity of the model were confirmed using Smart PLS 3.3.3 prior to data analysis and model testing. Partial least squares regression ‐ structural equation modeling (PLS‐SEM) was used for data analysis. Items with a variance inflation factor >5 were removed to control for common method bias.

Results: Both theories independently accounted for a considerable 66% of the variance in healthcare professionals’ security behavior. Perceived severity and response efficacy were found to positively influence avoidance motivation (AM) for PMT while all factors of the threat appraisal arm of TTAT and safeguard efficacy played a significant influencing role on AM in TTAT (p<.001). Response cost and self‐efficacy were not supported for both models, providing us with potentially new insights into this population’s security behavior.

Discussion: To the authors’ knowledge, this is the first study to compare TTAT and PMT in a healthcare context. Since the findings of this study might not be generalizable to other countries or contexts, future studies that focus on leveraging these theories to develop preventative measures in other contexts are warranted. An in‐depth understanding of which factors are significant (e.g., perceived severity) or not significant (e.g., response cost) between the two models could greatly benefit healthcare administrators and system designers alike in developing more robust tools targeting those measures to minimize security risk. The results from the current study collectively serve as a stepping stone towards future research in health information security as well as keeping the connected, digital health sphere secure.

4. Making a comeback from a global pandemic; a large telestroke programs experience from CY2020 compared to previous years

Aliza Brown PhD, Krishna Nalleballe MD, Renee Joiner RN, Lori Berry RN, Sanjeeva Onteddu MD

University of Arkansas for Medical Sciences

Background: During the pandemic calendar year (CY) of 2020, a large telestroke’s experience of consults treated with Alteplase were compared to previous years for number of consults, median time points and other time measures. We hypothesized that trending numbers of Alteplase treated consults increase annually.

Methods: The Institute for Digital Health & Innovation‐Stroke Program provides neurology support to 55 spoke hospitals across Arkansas. All spoke hospital sites are required to perform monthly mock scenario training. Stroke consults who received Alteplase during 01/01/20 to 12/31/20 were compared to previous CY’s since 2015. Retrospective analysis included median time for Door to MD (D2MD), Door to CT (D2CT), Door to Call Center (D2CC), Door to Neuro (D2Neuro) and Door to Needle (D2N). Number and percent of consults per year with percent Alteplase treated were compared.

Results: Pinnacle numbers in consults and percent treated occurred in 2019. Consult numbers declined in CY2020 vs CY2019 (1,289 vs. 1,460, respectively). Alteplase treated consults decreased from 44% to 41% in CY2019 to CY2020. Percent treated in </ = 60 min also declined from 53% to 50.7%. Median time in CY2020 for D2MD, D2CT, D2CC, and D2N were consistent with CY2019. An increase by two minutes in D2Neuro was seen in CY2020.

Discussion: Each year the IDHI‐SP telestroke has increased their number of consults and improved median time points. With the pandemic the telestroke program observed consult numbers reminiscent of 2018. While consult numbers dropped the program’s median time points remained consistent to 2019. Monthly mock consult training may have influenced the continued improvement in median times into CY2020 and warrant further analysis.

5. Minute to minute to win! Stroke care positively impacted by EMS

Aliza Brown PhD, Tim Vandiver RN, Lori Berry RN, Krishna Nalleballe MD, FAHA, Renee Joiner RN, Sanjeeva Onteddu MD

University of Arkansas for Medical Sciences

Background: Stroke care begins at the on‐scene triage by the emergency medical services (EMS) paramedics. From symptom identification to hospital pre‐notification coordination of care from scene to hospital should involve communication of EMS code stroke to hospitals and feedback for training on correct identification. We sought to determine rural EMS impact on timed stroke care with their area hospitals in one of the nation’s largest in‐state telestroke programs, the University of Arkansas for Medical Sciences (UAMS) Institute for Digital Health & Innovation‐Stroke Program (IDHI‐SP).

Methods: An analysis of time measures was performed from patients who arrived via EMS or privately owned vehicle (POV) in the UAMS IDHI‐SP from who received a consultation from the networks teleneurologists and tissue plasminogen activator (tPA) in 2019 (n = 647). The use of Code Stroke pre‐ notification and stroke bands was examined on the Door to CT (D2CT) and the Door to tPA (D2tPA) time measures using ANOVA and Fisher’s PLSD.

Results: In 2019, there were 31 missing entries for mode of transport (EMS or POV). EMS transports (n = 380) compared to POV transports (n = 236) on D2CT time was significantly improved 9.8 ± 0.6 vs. 18.4 ± 1 minutes, p<0.0001, respectively. D2CT when EMS pre‐notified (n = 321) 8.2 ± 0.5 vs no pre‐ notification (n = 50) 20.8 ± 3.2 minutes, was significantly improved p<0.0001. D2tPA time for EMS transports was improved at 61.6 ± 1.4 vs. POV transports at 68.8 ± 1.7, p = 0.0012. EMS pre‐notification significantly improved D2tPA 58.7 ± 1.4 vs. no pre‐notification at 80.6 ± 4.7, p<0.0001.

Discussion: Nurse facilitators at these sites benefited from early Code Stroke pre‐notification by EMS teams with dramatic improvements in treatment‐to‐care time. Continued training and repeated mock drills by the telestroke program with both EMS and spoke hospitals have benefited patient care.

6. Parent perspectives on use of telemedicine for pediatric acute respiratory tract infections

Kristin Ray MD, Tamar Krishnamurti PhD, Sarah Burns MSW

University of Pittsburgh

Background: The incorporation of telemedicine options into pediatric primary care creates the potential for both new opportunities and barriers. The degree to which child and parent preferences and larger social contexts influence family preferences for site of treatment for common childhood illnesses, such as acute respiratory tract infections (ARTIs), need to be better understood as telemedicine availability increases. The goal of this work was to understand factors parents consider when deciding to seek care for ARTI symptoms and how the availability of telemedicine impacts decision‐making.

Methods: One‐hour long, semi‐structured interviews were conducted with 40 parents of children between the ages of 1 and 5. Parents were asked questions about potential threats, barriers, trade‐offs, and preferences for seeking care. We qualitatively analyzed semi‐structured interviews with parents about their preferences, concerns and perceptions of trade‐offs when seeking care for children with cold symptoms.

Results: Forty parents participated in interviews. 43% of interviewees report living in a rural area, and 35% of interviewees had not used telemedicine for their children previously. Common themes when discussing decisions to seek care for pediatric ARTIs in general included severity of symptoms and ability to access care. Themes related to willingness to seek care via telemedicine for ARTIs primarily related to clinical factors (e.g., specific symptoms, suspected diagnosis), access factors (e.g., convenience), and estimations of the likelihood of telemedicine providing an adequate outcome (e.g., parental estimation of probability of need for follow‐up in‐person care, trustworthiness of telemedicine as a treatment modality, reliability of the clinician conducting the telemedicine encounter, and potential value of continuity in enhancing quality of telemedicine encounter). Many interviewees also volunteered greater readiness to us telemedicine for themselves than for their children.

Discussion: Parents weigh multiple factors when considering where to seek care for their children with cold symptoms. Telemedicine options are valued by parents when parents perceive that telemedicine has the potential to provide accurate evaluation and treatment, convenience, and reassurance.

7. Development of a Virtual Child Psychiatry Support Service to Support Rural and Remote Emergency Departments in Northern British Columbia (BC), Canada

Jennifer Begg RN,¹ Melissa Coop RN,² Kit Johnson MSc (Pharm),² Erica Koopmans MPH,² Anthon Meyer MD,³ Mahmoud Mitha MHA,² Angela Olsen MSW,⁴ Joelle Pellegrin RN,² Jennifer Russel MD,⁵ Marilyn Thorpe MD,⁶ Sina Waibel PhD,⁷ Linda Wu MHA²

¹Northern Health, Prince George, BC, Canada; ²Child Health BC, Provincial Health Services Authority, Vancouver, BC, Canada; ³Northern Interior Regional Division of Family Practice, Stuart Lake Hospital, Fort St. James, BC, Canada; ⁴British Columbia Children’s Hospital and Child Health BC, Provincial Health Services Authority, Vancouver, BC, Canada; ⁵Child, Youth and Reproductive Mental Health Programs, BC Children’s Hospital, Division of Child and Adolescent Psychiatry Program, Department of Psychiatry, University of British Columbia, Vancouver, BC, Canada; ⁶Division of Child and Adolescent Psychiatry Program, Department of Psychiatry, University of British Columbia, Vancouver, University of Victoria, BC, Canada; ⁷Department of Pediatrics, University of British Columbia and Child Health BC, Provincial Health Services Authority, Vancouver, BC, Canada

Background: In BC, pediatric visits to EDs due to mental health or substance use (MHSU) are increasing. Physicians and staff have identified a need for increased specialty support in caring for children and youth with MHSU concerns. Few child/youth (C&Y) psychiatrists in rural, remote, northern communities makes access to timely and appropriate mental health care challenging. A virtual system is needed to begin to address these acute mental health care concerns particularly where equitable access continues to be the primary challenge.

To approach this challenge, a collaboration among stakeholders and agencies at the local, regional, and provincial level was required to develop an evidence‐based service model. The model is intended to augment the existing services offered to providers by offering a virtual service after business hours. A MHSU education component will complement the model with the focus on increasing confidence and competencies of healthcare providers caring for C&Y admitted to EDs.

Methods: A small rural hospital in Northern BC with approximately 700 annual pediatric Emergency Department (ED) visits was selected as a demonstration project. Three different types of evidence were used to inform service design which was based on the Child Health BC Mental Health Tiers of Service module and self‐assessment. First, an environmental scan and a literature review was conducted to understand existing virtual services and identify service models used by other jurisdictions. Second, data from Northern Health and BC Children’s Hospital were analyzed to gather information about ED service volume and use of existing C&Y MHSU services. Third, key informant interviews were conducted with local ED service providers (n = 6), MHSU and community providers (n = 13), and C&Y psychiatrists (n = 9). Thematic content analysis was completed to understand the current state and determine service needs. A series of stakeholder meetings were held to discuss and vote on key components of the service based on the collected evidence, and a virtual service model was finalized.

Results: The environmental scan identified virtual visit technology to complete direct patient assessments and provide clinicians with virtual peer‐to‐peer support to deliver care locally. Local data reported the highest volume of C&Y ED visits for MHSU concerns occurred between 16:00‐22:00, 7 days a week. Interviews highlighted challenges such as lack of standardized procedures and communication protocols during transitions to higher levels of care or discharge to community. The resulting service model is a provider‐to‐provider consultation with a virtual team comprised of a C&Y psychiatrist, allied C&Y MHSU professional, and patient navigator liaison, available seven days a week from 16:00– 23:00. The service includes: (1)Virtual consultation, including guidance on diagnostic clarification, medication recommendations, and treatment planning (2)Navigation of local and online resources (3)Education customized for the situation (4)Optimization of existing resources (5)Clear referral protocols to achieve longitudinal continuity (6)Facilitate service referral/connection to adult mental health services for requests of support regarding youth over the age of 18.99 years (7)Implementation of referral algorithms to transfer to higher level care.

Discussion: The model aims to provide timely access to EDs for urgent, virtual psychiatric specialist services to meet the needs of C&Y and their families within their own community, and to build MHSU knowledge and capacity of local providers. Ongoing quality improvement monitoring using key performance indicators will be conducted. These indicators will be linked back to the provincial Mental Health Quality Indicators. A further evaluation framework will be established in collaboration with the local, regional, and provincial stakeholders to evaluate the service model’s short and long‐term outcomes. Quality of care will be assessed using dimensions established through the provincial quality and safety council (respect, safety, accessibility, appropriateness, effectiveness, equity and efficiency), to understand the performance of the service model from both the patient and provider perspective. A scale up of the service model to other sites across BC will be considered, following evaluation.

8. Branching Out: Using STEM to Assess the Impact of a Pediatric Sleep Medicine Clinic’s Transition to Virtual Care

Innessa Donskoy MD, Matt Balog MPH, Darius Loghmanee MD, Kathleen Webster MD

Advocate Aurora Health

Background: Sleep plays an integral role in a child’s physical and psychological development, and a well‐rested child thrives in academic and social interactions. Alternatively, a child struggling with falling asleep, staying asleep, or excessive daytime sleepiness may present with challenges in these areas. There is a paucity of pediatric sleep medicine providers equipped to partner with families in addressing these challenges, and accessing this care requires time and geographical proximity. Ensuring these clinics run efficiently is paramount for providing as many children access to sleep care as possible. Before the SARS‐CoV‐2 pandemic, our pediatric sleep department was predominantly in‐person with a small pilot of telemedicine follow‐up visits available. After the stay‐at‐home order was issued in the United States, our clinic transitioned to an exclusively virtual platform.

Methods: The STEM framework1 was developed by pediatric telehealth experts in order to provide a unified approach to assessing the impact of telehealth programs. We applied this framework to available clinical and quality data to assess impact on 1) health outcomes via past medical history and sleep disorders managed; 2) health delivery using visit volume, show rates and geographic distance from the nearest clinic site 3) Individual experience assessed by provider satisfaction and patient willingness to perform visit via telehealth and 4) program logistics as demonstrated by number of available appointments, percent of visits conducted via telehealth and electronic tools utilized to enhance patient compliance.

Results: Data from our pediatric sleep medicine clinic from June 2019 to June 2021 will be presented, demonstrating a greater number of available appointments and higher clinical efficiency when care was provided via a virtual platform. The patient distribution differed from in‐person visits, namely with more children presenting with behavioral sleep issues and located further away from in‐person clinic sites.

Discussion: Pediatric sleep care can be challenging to access. Physical distance and time away from school and work, areas where a child may already be struggling from poor sleep, preclude many families from formal sleep consultations. Long commutes also limit providers in how many patients they can see. When care is provided virtually, more time can be carved out for the clinical day. With a convenient click, families can fit sleep consultations easily into their day without the stress of travel or the waiting room. They are also more likely to seek care for behavioral sleep issues like insomnia, which can impact sleep health as much as physical conditions like sleep‐disordered breathing. Electronic reminders and last‐minute links allow a quick entry into the visit and rarely missed appointments.

Telemedicine allows high‐quality pediatric sleep care to be offered with more availability in a format more easily accessible to those near and far, resulting in clinics running more efficiently and more children receiving the tools they need to sleep soundly.

9. Telemedicine Experiences in Developmental‐Behavioral Pediatrics: A DBPNet Study of Providers and Caregivers

Kate W. Wallis MD,¹ Katie Kellom BA,² Audrey Christiansen MD,³ Lucero Cordero BA,¹ Julia Hah MSW,⁴ Kristen Stefanski MD,⁵ Priscilla Ortiz PhD¹

¹The Children’s Hospital of Philadelphia ²The Children’s Hospital of Philadelphia, PolicyLab; PolicyLab, ³Boston University/Boston Medical Center ⁴Perelman School of Medicine at the University of Pennsylvania, ⁵Akron Children’s Hospital

Background: To meet patient needs during the COVID‐19 pandemic, >90% of academic developmental‐behavioral pediatric (DBP) practices expanded telehealth use, including video evaluations of young children with possible autism spectrum disorder (ASD). Little is known about the acceptability of telehealth assessments among providers and families, or perceived benefits and challenges. This study aims to inform the use of telehealth in DBP care with an emphasis on equity.

Methods: Three academic DBP practices are included in this ongoing, mixed‐methods study of DBP providers and caregivers. Geographically diverse sites were selected through DBPNet, a national research network of 16 DBP practices. DBP providers and caregivers who participated in new‐patient telehealth visits since 3/1/2020 assessing for possible ASD in children <5 years‐old at participating sites are eligible. Electronic surveys collect participant demographics, comfort using technology (caregivers) and diagnosing ASD (providers), preferred use of telehealth, and responses to the standardized Telehealth Usability Questionnaire. Caregiver materials are available in English and Spanish. Open‐ended interviews with providers and caregivers ask about experiences with telehealth, perceived benefits and challenges, confidence in accuracy of the assessment, future uses of telehealth, access to DBP care, and the impact of telehealth on equity, including for families with limited English‐proficiency. Interview transcripts are reviewed using content analysis. Recruitment will continue until thematic saturation is reached.

Results: To date, 9 DBP providers completed interviews. All providers report completing at least 5 telehealth visits (new or follow‐up) on average each week since March 2020. For English‐speaking families, all providers report being very comfortable diagnosing ASD for in‐person assessments, and very (6/9), somewhat (2/9), or neither comfortable or uncomfortable (1/9) diagnosing ASD by telehealth. For Spanish‐speaking patients, providers report feeling very (4/9) or somewhat (5/9) comfortable diagnosing ASD in‐person, and very (3/9) or somewhat (6/9) comfortable by telehealth. Interviews describe additional challenges using interpreters and building rapport without body language via telehealth. Most providers wish to continue using telehealth for new and follow‐up visits in the future.

To date, 8 caregivers have completed the survey and 6 have been interviewed (all English‐speaking). Families reported high satisfaction with their DBP telehealth visit; 7/8 would use telehealth again for DBP care. Many families reported a desire to seek follow‐up care by telehealth, but varied in their acceptability of virtual assessments post‐pandemic. Telehealth systems are reportedly usable, with higher ratings of usability by providers than caregivers.

Discussion: Caregivers and providers are generally content with virtual ASD assessments, but family confidence in the quality of assessments varies. Concerns about digital equity and language barriers remain. Emerging themes include acceptability of telehealth visits, challenges of telehealth, role of the home environment, equity and access to telehealth service and future use of telemedicine. Additional analyses will further elucidate caregiver and provider beliefs about telehealth use in DBP to guide best practice shaping future care.

10. Patient, Provider, and Site Characteristics Associated with Telehealth Service Use at VA Greater Los Angeles Healthcare System during COVID‐19

Claudia Der‐Martirosian PhD, Aram Dobalian JD

US Department of Veterans Affairs

Background: Use of real‐time synchronous video technologies, such as VA Video Connect (VVC) and clinical video telehealth (CVT), has been increasing at the US Department of Veterans Affairs (VA) in the past two decades. VVC, first piloted in 2017, is the primary platform for video conferencing telehealth modality approved by VA. VVC, unlike CVT, does not require patients to travel to the nearest VA facility [a community‐based outpatient clinic (CBOC)]. VVC helps connect Veterans with their health care providers via secure/private sessions for live, video‐on‐demand visit appointments to patients’ homes. VA telehealth has previously been expanded during major disasters, such as hurricanes, but it was not until COVID that there was a need for widespread, rapid expansion of real‐time video telehealth in primary care (PC). There is a need to better understand patient, provider, and site characteristics associated with telehealth services use, especially video‐based care.

Methods: We used VA administrative and clinical data to examine patient, provider, and site predictors of any virtual care (phone or video vs. in‐person), and video‐based care (video vs. phone) in PC at the Greater Los Angles Healthcare System (GLA) VA. We used individual‐level, interrupted time series (ITS) analysis through segmented logistic regression on repeated monthly observations of any virtual care or video‐based care over 24‐months (3/1/2019‐3/1/2021), 12‐months before and 12‐months after onset of COVID. ITS was divided into four segments: 1) pre‐COVID, 2) onset of COVID‐19 (stay‐at‐ home orders initiated, March 2020), 3) re‐expansion of in‐person services at GLA and lifting of stay‐at‐ home orders, 4) the 2020‐2021 flu season. The ITS multilevel analyses included patient‐ and provider‐ level clustering and adjusted for patient characteristics: age, gender (male/female), race/ethnicity groups, marital status, non‐VA health insurance coverage (yes/no), health risk factors, homeless (yes/no), service‐related disability/income status; type of provider; site type (medical facility vs. CBOCs). The study cohort/sample size included 547,730 visits, 64,361 patients, 1,330 providers, and 2 site types.

Results: Before onset of COVID, any virtual care (VC) in PC at GLA varied 16%‐20% per month; majority were telephone encounters. At the onset of COVID, use of telehealth increased substantially (up to 60%). Patterns of VC corresponded to the initiation and lifting of stay‐at‐home and social distancing mandates in Los Angeles, CA in the first 12‐months of COVID. The racial/ethnic distribution for PC patients at GLA is 43% White, 21% non‐Hispanic African American, 18% Hispanic. Hispanics, compared to non‐Hispanic Whites, were less likely to use VC (OR = 0.93;95%CI:0.89‐0.96) or video (OR = 0.85,95%CI:0.77‐0.93). Non‐Hispanic African Americans compared to non‐Hispanic Whites had a marginally increased likelihood of using VC (OR = 1.09,95%CI:1.05‐1.13) but a decreased likelihood of using video (OR = 0.76,95%CI 0.69‐0.83). Among younger Veterans (18‐44, 45‐64, 65‐74), women were more likely to use VC or video than men. Compared to MD/NP/PA, all other healthcare professionals, except for LVN/MSA, were more likely to use VC; yet, all PC professionals, except MH, were less likely to use video. Compared to GLA community‐clinic patients, Veterans who received PC services at the main medical facility were less likely to use VC but more likely to use video.

Discussion: Implementation of complex healthcare delivery methods, such as the rapid expansion of telehealth services at a large integrated healthcare system (VA), requires evaluation at all levels: patient, provider, site. Identifying racial/ethnic, gender, and age disparities in VC/video use can help us understand the digital divide and how access to VC can be improved for all patients. Greater understanding of which types of PC providers are more/less likely to use VC/video, and which types of PC services are better suited for VC, can guide integration VC in clinical practice. Site type differences may allude to various factors affecting VC/video use, such as urban vs. less urban or rural areas, or there may be differences in site infrastructure, support, or resources for VC. This multi‐level assessment can help standardize implementation of telehealth, especially video‐based care, to maximize efficiency, increase access to care, improve quality of VC, and facilitate scale‐up in PC.

11. Telemedicine for Pediatric Asthma Care: Adoption During COVID‐19 and Implications for the Future

Sarah C. Haynes PhD, Rory Kamerman‐Kretzmer MD, Shahabal S. Khan MPH, Stephanie Crossen MD, Monica K. Lieng PhD, James P. Marcin MD, Nicholas J. Kenyon MD, Christopher H. Kim MD

UC Davis Health, UC Davis School of Medicine

Background: Prior to the COVID‐19 pandemic, telemedicine for routine asthma care was uncommon. Since the beginning of the pandemic, however, there has been a drastic increase in the use of telemedicine visits for asthma. While telemedicine presents an opportunity to increase access to asthma care, little is known about which patients have successfully adopted telemedicine and how patients and families experience telemedicine visits for asthma care. The purpose of this study was to better understand the adoption of telemedicine for asthma care among pediatric and young adult patients during the pandemic to inform the future of care using telemedicine for asthma in this population. To this end, we conducted a mixed methods study comprising an electronic health record (EHR) analysis and a qualitative focus group analysis.

Methods: We abstracted data from the UC Davis EHR on all patients ages 2‐24 who had any visit with asthma as the primary diagnosis in the six months following the California shelter‐in‐place orders (March 19, 2020 – September 30, 2020). We fit a multivariable logistic regression model to calculate the odds of telemedicine use and 95% confidence intervals. We examined the association of telemedicine use with sex, age, race, insurance, primary language, primary vs specialty care, rural/urban residence, and distance to UC Davis Health. We then conducted focus groups with a sample of the participants from the EHR analysis. Participants were eligible for the focus groups if they were a parent of a patient aged 2‐17 or a patient aged 18‐24 who had a visit with a primary diagnosis of asthma during the study period. We excluded parents and patients who did not speak English. Focus groups lasted one hour and were conducted over Zoom; participants also completed a brief survey to report demographic information. Focus group transcripts were coded independently by two coders using a combination of structural codes and process codes. Themes and categories were reviewed and organized by the entire research team.

Results: A total of 502 patients aged 2‐24 were seen for asthma care at UC Davis Health during the pandemic, as defined by a visit with a primary diagnosis of asthma. 207 of these patients (41.2%) had at least one telemedicine visit. Telemedicine use was significantly lower among patients with a primary language other than English (OR = 0.12, 95% CI: 0.025‐0.54, p = 0.006), school‐aged children (OR = 0.43, 95% CI: 0.24‐0.77, p = 0.005), and patients who received asthma care only from a primary care provider (OR = 0.55, 95% CI: 0.34‐0.91, p = 0.020). We conducted three focus groups in April and May of 2021, after which the research team determined that thematic saturation had been reached. Focus groups comprised 12 parents of pediatric patients and 5 young adult patients. Focus groups participants identified several important challenges and opportunities facing telemedicine for asthma care, including establishing therapeutic alliance over telehealth, engaging pediatric patients, using home monitoring tools to supplement telehealth visits, and scheduling of follow‐up appointments. Nearly all participants across the three focus groups agreed that a combination of telemedicine visits and in‐person visits would be preferred when asthma is well controlled.

Discussion: Our findings suggest that alternating in‐person and telemedicine visits for routine asthma care is acceptable to parents of pediatric patients and to young adult patients with asthma when asthma is well controlled. Using telemedicine and in‐person visits in conjunction may be an effective way to balance the improved access and convenience offered by telemedicine with the improved patient experience and measurements offered by in‐person visits. Although the use of home monitoring devices such as spirometers and peak flow meters may complement telemedicine visits, our study findings indicate that there may be reluctance among many patients and parents to use these devices due to low self‐efficacy and high perceived burden. Efforts should be made to identify and address telemedicine access barriers for patients with a primary language other than English to ensure that a shift towards telemedicine does not contribute to widening health disparities.

12. Implementing Telehealth and Image Exchange to Create a Virtual Pediatric Trauma Center

James P. Marcin MD, Joseph Galante MD, Jennifer Lynn Rosenthal MD, Tanya Rinderknecht MD, Kendra Grether‐Jones MD, Michelle Hamline MD, Marike Zwienenberg MD, Brian M. Haus MD, Kristin Matthews MSN, Katherine Rominger RDN, April Sanders MPH, Siedah Garrison MPH, Raynald Dizon, BS, Hadley Sauers‐Ford MPH, Nathan Kuppermann MD

UC David Health

Background: Using telehealth to connect specialists to rural and community hospital emergency departments (EDs) has been successful in the management of a variety of conditions including stroke, mental health emergencies, and pediatric critical illnesses. Under our PCORI‐funded trial, we have begun an evaluation of a “virtual pediatric trauma center” (VPTC) in which pediatric trauma, orthopedic, and neurosurgery providers use videoconferencing and image sharing to initiate care of injured children remotely in a receiving hospital ED. The goal of the study is to compare the current standard of care to the VPTC model of care with regards to the parent/family experience of care, parent/family distress, healthcare utilization, and out‐of‐pocket cost burden. The study enrollment began in November 2020 and will continue until November 2022. In this abstract, we report our implementation and operation processes as well as lessons learned in implementing the VPTC among a diverse group of EDs and providers

Methods: The aims of the study are to compare the current standard of care to the VPTC model of care with regards to: 1) the parent/family experience of care and distress 3, 30, 60 and 90 days following a childhood injury; 2) the 30‐day healthcare utilization following the injury event; and 3) the out‐of‐ pocket costs and financial burdens experienced by parents/families 3‐days and 30‐days following the injury and ED visit to a non‐pediatric trauma center. To accomplish the aims of this study, we implemented a stepped wedge trial design among a stratified selection of 10 hospital EDs in Northern California, with a goal of enrolling 380 patients by November 2022. To operationalize the VPTC model of care, we deployed pole mounted videoconferencing units with high‐definition monitors, omnidirectional microphones, and remote‐controlled pan‐tilt‐zoom 15x optical zoom cameras.

Pediatric trauma providers at the VPTC were provided access to computer workstations as well as video capable mobile devices. We also incorporated clinical back‐up from a team of trauma Nurse Practitioners (NPs) at the receiving ED to be available for videoconferencing and are using an image sharing platform that leverages server‐based and cloud‐based solutions.

Results: To date, 122 pediatric trauma patients have been enrolled. Protocol adherence for videoconferencing has occurred in 18 of 43 eligible patients; adherence for image sharing has occurred in 16 of 38 eligible patients. Collection rates of surveys of parent/family experience of care and distress and financial burden surveys has been 65%. The ability of the trauma, orthopedic and neurosurgery providers to connect using telemedicine has been limited by their other clinical responsibilities, and the protocol relies heavily on the NPs for administering telehealth communications and recommendations. The study team (including a parent coinvestigator) has maintained at least weekly meetings, in addition to quarterly meetings with a community advisory council and a family advisory board. Several changes in the workflow processes were made to increase the reliability and fidelity of the interventions and data collection, which will be shared in the poster/presentation.

Discussion: Implementing telehealth and image sharing in the delivery of acute pediatric trauma care is challenging. Challenges include incorporating videoconferencing by clinically busy providers and implementing project‐specific platforms at partner sites. Incorporating telehealth and image sharing into existing workflows and obtaining “buy‐in” from several pediatric specialty teams is a significant investment in culture change. However, with this commitment, acutely injured children can receive regionalized pediatric trauma expertise at the bedside in a receiving hospital ED. As the project continues, the research team will continue to identify challenges in implementing telehealth technologies in the ED setting. Current findings illustrate the need for qualitative data to improve the family experience. We will explore interviewing providers and families throughout the study to continue the evaluation of the clinical workflow and the telehealth experience for acute trauma care.

13. Telemedicine for Pediatric Asthma Care: Adoption During COVID‐19 and Implications for the Future

Sarah C. Haynes PhD, Siedah Garrison MPH, Amanda Favila‐Meza BS, Jeffrey S. Hoch PhD, Daniel J. Tancredi PhD, Raynald Dizon BS, James P. Marcin MD, Loren Davidson MD

UC Davis Health

Background: Children with acquired or congenital disability who live in rural communities have unique medical and therapy needs. Pediatric physiatrists (subspecialist physicians with expertise in caring for children with disability), are scarce and are concentrated in urban areas. In California, the California Children’s Services Medical Therapy Program guides the therapy and comprehensive medical management of children with special health care needs at school‐based clinics called Medical Therapy Units (MTUs).

Telemedicine may reduce burden on rural families by allowing physicians to provide care in the communities in which the patients reside. Our novel tele‐physiatry program allows pediatric physiatrists to provide direction over video while a therapist is present with the patient. The aim of this study was to evaluate the difference in parent experience and perceived quality of care between a traditional school‐based clinic with an in‐person physiatrist against a telemedicine‐based approach.

Methods: We designed a non‐inferiority, cluster‐randomized crossover study at four school‐based clinics to evaluate parent experience and perceived quality of care between in‐person and telemedicine‐based approaches. In the intervention, the physiatrist directed encounters over video while a physical and/or occupational therapist performed the clinical exam. The connections use a video software solution to connect the physiatrist to a pole‐mounted monitor‐computer equipped with a high definition video camera with remote pan‐tilt‐zoom capabilities. Physiatrists would perform exams in‐ person at the MTU for control encounters. Physiatrists recruited parents for the study and obtained verbal consent. Parent surveys included five questions adapted from the Consumer Assessment of Health Providers and Systems Hospital Survey. We summarized the parent experience score by normalizing each of the five questions to a 0 to 1 scale, according to the lowest and highest response choices, and then averaging the five normalized scores together. We used regression analyses to determine the adjusted mean difference in the parent experience summary score between telemedicine and in‐person encounters adjusting for patient age, race, calendar year, and site.

Results: A total of 268 encounters (124 telemedicine and 144 in‐person) were completed by parents of 200 unique patients. The average age of the children was 10.9 years. The sample was diverse with respect to race and ethnicity; 23.7% of children were white, 10.2% were black, 23.7% were Hispanic, and 11% were Asian. 68.7% of children had a diagnosis of cerebral palsy. A total of 11 therapists (9 physical therapists, 2 occupational therapists) completed 268 surveys for encounters with 200 unique patients. The experience and perceived quality of care were high with no significant differences between telemedicine and in‐person encounters (Adjusted Mean Difference: ‐0.09, 95% CI: ‐0.20, 0.03, p = 0.15). For parents whose children received a telemedicine encounter, 54.8% reported no preference for their child’s subsequent encounter, 28.8% preferred a physiatrist telemedicine visit, and 16.4% preferred a physiatrist in‐person visit. Therapist confidence in the quality of care provided did not differ by encounter type (Adjusted Mean Difference: ‐0.02, 95% CI: ‐0.35‐0.30, p = 0.89).

Discussion: We found that a school‐based tele‐physiatry model of care is not inferior to standard, in‐person physiatry encounters regarding patient and therapist experience and perceived quality of care. This finding addresses concerns that stakeholders had when telemedicine models of care have been proposed as an alternative to in‐person care. Positive parent and clinician experience of care and satisfaction with quality in our study align with previous studies on provision of care via telemedicine for children with special healthcare needs. More importantly, our tele‐physiatry model of care involves therapists being present with the patient and performing a comprehensive physical exam.

This model of tele‐physiatry is unique and the first to our knowledge to be reported. Our study is resulting in increased access to specialty pediatric physiatry care for children with special healthcare needs in Northern California; our findings provide support for an expansion across the state and the nation.

14. Pediatric, Urgent Care Telemedicine during COVID‐19: Pivots, Patients, and Potential

Cynthia Zettler‐Greeley PhD, Joanne Murren‐Boezem MD, Patricia Solo‐Josephson MD

Nemours Children’s Health

Background: COVID‐19 increased use of virtual health care. Likewise, the virus transparentized enduring inadequacies in access to care within the U.S. CDC reports that Black and Hispanic adults are disproportionately infected and are 2.8x more likely than Whites to require hospitalization, a pattern that holds for children. Such findings are embedded in social determinants of health, imploring re‐ examination of how to improve access to health care for all. Telemedicine holds this potential. But to what extent has it been realized among those at greatest risk? This retrospective study describes a pediatric, urgent care platform from the call to close office doors to the redeployment of providers to assist in delivering virtual urgent care, from the height of the lockdowns to the present. We describe patients who are most adversely impacted by the pandemic to elucidate where efforts are working and discuss where further efforts must be made to support access to care, during the pandemic and beyond.

Methods: Nemours Children’s Health launched telemedicine in 2015, serving pediatric patients with urgent care, specialty care, and primary care in the Delaware Valley and Florida. Six board‐licensed pediatricians operating the 24/7 urgent care platform treat health complaints, emergency room follow‐ups, and many others. During March —May 2020, peak utilization for completed urgent care telemedicine encounters increased by 226% relative to the same months in the prior year as medical facilities closed for in‐person visits. The dramatic volume increases led to the rapid training and credentialing of 28 additional providers to staff the platform, facilitated by relaxed federal and state restrictions. In this study, we extend recently published work that illuminated a more diverse patient population utilizing urgent care telemedicine from January—June 2020 relative to the same months in the prior year. As telemedicine use moderates from those peak levels, we examine patient age, sex, race, ethnicity, language, rurality, income level and insurance type into the second year of the pandemic to determine whether previously observed changes in patient diversity persist as telehealth utilization delves into a new normal.

Results: Since May 2020, 11,123 patient‐families have utilized Nemours urgent care telemedicine. In this study, we examine demographic characteristics among these patients across Nemours’ two primary markets. Patient data for completed telemedicine visits extracted from the electronic health record between June 2020—July 2021 will be used to address the following research questions:

• To what extent is virtual care being utilized by patients most at risk of poor health outcomes due to factors intertwined with social determinants of health (e.g., race, ethnicity, language, income, rurality)?

• To what extent is the demographic diversity among patients observed during peak telemedicine utilization early in 2020 maintained currently, when patients again have the option of appointments in medical facilities?

• Is there a new demographic “normal” of pediatric telemedicine utilizers that is evident in Year 2 of the pandemic?

• Do differences in telemedicine utilization continue to be found across Nemours’ two primary markets?

• Are there significant demographic differences among patients utilizing telemedicine across pre‐pandemic (e.g., 2019), peak pandemic (March‐May 2020), and current (June 2020‐July 2021) timeframes?

Discussion: This research examines demographic changes among patients utilizing telemedicine and considers how to meet the virtual care needs of a more diverse patient population. Pandemic‐driven increases in telemedicine access and utilization continue to illuminate its relevance as a key modality of the care continuum, even while many patients remain at risk for poor health outcomes due to social and economic factors. Examining whether patients at increased risk have benefited from greater access to telehealth will highlight opportunities for healthcare systems and policy makers to act in order to realize the potential that telehealth offers, whether by technological outreach to patients in rural locales or expanding translation services to meet increased linguistic diversity. The extent to which telemedicine’s potential can be realized has profound implications for access to care, while reducing negative impacts of social determinants of health, among an increasingly diverse patient population.

15. Using Machine Learning to Advance Telehealth

Daniel Liu MD, Tamara Perry MD, Pele Yu MD

University of Arkansas for Medical Sciences

Background: The COVID‐19 pandemic brought rapid changes in telemedicine adoption. Many institutions saw a rapid surge in telemedicine utilization followed by stabilization at a lower level, between 17‐25% depending on the region. For our institution, telemedicine visits stabilized at a level below regional and national averages (∼5%). Machine learning has been used in healthcare scheduling to predict and even reduce no‐shows but has not been applied to classifying visit types. As there have been legal and technological changes that make telemedicine a better proposition for delivering healthcare, we are looking to expand our telemedicine utilization sustainably. A machine learning model that could use patient and visit characteristics to classify potential telemedicine visits accurately could help achieve this goal.

Methods: The IRB determined this project to be non‐human subjects research. A de‐identified dataset of patient and visit characteristics was obtained. After feature selection, the model includes a label of telemedicine vs. office visit; features of patient age, race, sex, ethnicity, first 3 of zip code; visit department specialty; and time‐related variables. The training‐testing data split was 80‐20. As telemedicine visits were only three percent of the combined visits, random and SMOTE‐NC oversampling methods were compared for balancing the data.

Logistic Regression, Random Forest models were chosen for baseline comparison. Gradient‐boosted decision tree models were found to perform well in the healthcare scheduling domain, so XGBoost and Catboost were selected as well. The models were fit to the data then compared and evaluated using precision and recall. Precision and recall were chosen as incorrectly classified visits would lead to provider/patient dissatisfaction and missed expansion opportunities. Models underwent hyperparameter tuning via cross‐validation and feature importance analysis using mean SHAP values (avg impact on model output magnitude). The models were also evaluated using only inter‐pandemic data.

Results: The dataset showed demographic differences between pre‐ and inter‐pandemic usage. African American percentage utilization of telemedicine increased, whereas Hispanic or Latino use decreased. Both remain smaller than their respective percentages of our overall patient population.

Models fit using SMOTE‐NC balanced‐dataset performed better than the same using random oversampling. Logistic Regression produced the highest recall at 97%, but with a precision of 10%. The F1 score was 0.18. However, it was found to classify visits largely based on time. Any visits after the pandemic began were classified as telemedicine. XGBoost performed similarly with a precision of 13%, recall of 93%, and F1 of 0.22.

Random forest and Catboost performed similarly with higher precision (66% RF, 77% Cat) but lower recall (39% RF, 48% Cat). Catboost produced the model with the highest F1 score of 0.59; random forest’s F1 was 0.49. Catboost remained the best when using only inter‐pandemic data, with a precision of 0.78, recall of 0.46, and F1 of 0.58. The most important features for the Catboost model were zip, department, race, with SHAP values of ∼2, 1.65, and 1.

Discussion: The dataset showed disparities in telemedicine utilization. A more in‐depth analysis of race and ethnicity variables is planned for the next version of the model, especially if they remain significant features. Model inputs could be weighed so that the model considers each group equal. These disparities may also be better addressed by outreach efforts or removing barriers to telehealth specific for these populations.

The best machine learning model had s precision comparable to other machine learning models for healthcare scheduling. However, the recall was low, which means missed expansion opportunities. The next step is to add more data to the model, including patient‐level variables for different types of telemedicine use, the reason for visit, and insurance payer, which should improve performance.

Conclusion: The best machine learning model produced an acceptable precision score, but recall must be improved before considering implementation.

16. Early Insights: Patient and Provider Telehealth Experiences During COVID‐19

Mary Dooley PhD, Amanda James BSN, Kit Simpson DrPH, Natasha Ruth MD, Whitney Marvin MD, Kathryn King MD, James McElligott MD, Jillian Harvey PhD

Medical University of South Carolina

Background: Prior to 2020, telehealth visits were commonly acknowledged as popular with providers and patients yet contributed relatively small volumes to overall care delivery. However, COVID‐19 drastically changed how healthcare is delivered. When the pandemic struck, telehealth visits became essential for both patients and providers. Traditional outpatient appointments were rapidly converted to video visits to maintain the access and continuity of care. As we move forward it is expected that telehealth visits will find a new balance: lower than pandemic levels, but greater than pre‐2020. This change means it is important to understand the patient and provider experiences with telehealth services during this exponential growth and incorporate lessons learned. The objective of this study is to examine the virtual care experiences of patients and providers during COVID‐19.

Methods: We used a multi‐methods examination of patient and provider experiences. Patient‐level experiences were collected using visit data from pediatric in‐person visits in October 2019 and pediatric telehealth visits in October of 2020 from Press Ganey Patient Surveys. Provider‐level experiences were collected using 8 focus groups held via Teams with Pediatric Divisions. Content analysis was performed with initial coding using pre‐defined categories based on theoretical frameworks from the literature. New thematic categories were created to capture emerging themes.

Results: The analysis included 229 in‐person and 56 telehealth patients. Age and sex distribution was similar but Medicaid and commercial insurance were inversely proportionate compared to their in‐person visit distribution (p = 0.002). Patient reported willingness to recommend the visit differed;75% in‐ person patients strongly agreed compared to 49.1% for telehealth. All telehealth responses included written comments and 79.5% of in‐person patient had comments. There were different factors driving patient experiences across settings; in‐person visits focus was on timeliness (e.g. swiftness of scheduling an appointment, visit wait time) and patient‐centered care, however, telehealth visits focus was on efficiency (e.g. technology quality, technology ease of use, convenience). The focus groups had 83 providers. Discussion focused on factors that enhance physician well‐being and finding new norms and balances. Providers perceive telehealth and enhanced virtual options during COVID‐ 19 as enhancing their well‐being. One respondent said, “I actually get to eat lunch because we have our Telehealth schedule. When we are in person, we don’t have a break. So, the fact that I can actually eat lunch. It’s great, I think there’s a huge benefit.”

Discussion: Qualitative data provide rich insight into the drivers of patient satisfaction‐these data should not be overlooked. Due to the rapid transition to telehealth during COVID‐19 there is a need for process improvement and standardization of norms for both patients and providers. Different factors drive patient experience in in‐person vs. telehealth visits. The telehealth concerns were primarily related to technology and audio issues, thus, after technology issues are resolved researchers can explore if patient satisfaction will surpass in‐person or will patients move on to higher‐order issues, such as patient centeredness. Telehealth is a promising way to improve physician well‐being, however, ensuring healthy work/life integration and maintaining interprofessional teams and relationships needs to be addressed. Telehealth satisfaction should continue to monitor to understand if drivers or outcomes change once the technological barriers are overcome.

17. A modification of time‐driven activity‐based costing for comparing cost of telehealth and in‐person visits

Mary Dooley PhD, Kit Simpson DrPH, Annie Simpson PhD, Paul Nietert PhD, Dunc Williams PhD, Kathryn King MD, James McElligott MD

Medical University of South Carolina

Background: When the COVID‐19 pandemic struck, telehealth and virtual visits (TH) became essential for both patients and providers. The urgent need to convert to TH meant that few health systems had time to make deliberate choices between TH modalities. Programs in place locally were scaled up and improvisations were common. Organizations must now make strategic decisions to operate a streamlined, sustainable TH approach, which involves identifying best‐practice opportunities for improvement. The cost and value of TH services developed during the pandemic scale‐up should inform our choices. Largely missing from the literature are studies that identify the cost of delivering telehealth versus in‐person care from a provider perspective. The objective of this project was to demonstrate the use of a modified TDABC approach to compare the weighted labor costs of an in‐ person pediatric clinic sick visit before COVID‐19 to the same in‐person and telehealth clinic sick visit during the COVID‐19 pandemic.

Methods: We examined visits before and during COVID‐19 using: 1) recorded structured interviews with providers, 2) iterative workflow mapping, 3) EHR time stamps for validation, 4) standard cost weights for wages, and 5) clinic CPT billing code mix for complexity weighs. We examined the variability in estimated time using a decision tree model and Monte Carlo simulations.

Results: The clinic process for an in‐person visit during COVID‐19 for a non‐COVID risk patient was almost identical to the process before COVID‐19. However, the in‐person clinic visit with a COVID risk resulted in less overall labor time due to fewer actors involved. The estimated labor time for the process is 18 to 19 minutes, regardless of the method of delivery or the effect of the pandemic. Using TDABC, the labor costs for in‐person visits before and during the pandemic were similar; however, the process for a TH visit had slightly lower labor cost ($49.61 vs. $54.68 for MD and $31.63 vs. $36.75 for NP, for TH and in‐person visits, respectively). Using Monte Carlo simulations to examine the effect of variations in minute and salary cost estimates, assuming an 80/20 mix of MD and NP providers, and 28.3/71.7 mix for TH and in‐person visits during COVID‐19, the weighted cost of clinic labor for a sick visit was 6% lower during COVID‐19 than the cost of the same visit before COVID‐19 ($51.55 vs $54.47).

Discussion: As policymakers at all levels and healthcare executives continue to face decisions about telehealth versus in‐person care delivery in the rapidly evolving environment, our modification to TDABC may inform discussions about: 1) which TH programs to keep; 2) how to improve TH efficiency; and 3) which factors in a clinic’s workflow can be changed to achieve the most efficient mix of TH and in‐ person visits. Our TDABC approach is feasible to use under virtual working conditions, requires minimal provider time, can be implemented quickly, captures important variations in the process of care that affect costs, and generates detailed cost estimates that have “face validity” with providers that are relevant for process improvement and economic evaluation.

18. Promoting Health Care Equity in Telehealth through Provider Training

Sarah Hampton BA

Association of American Medical Colleges

Background: Many questions remain unanswered about the impact of telehealth on health equity and its ability to improve access for all. For many at the margins, there are numerous barriers to access—lack of broadband, excessive costs for services and devices, lack of comfort with technology, language and cultural barriers, a lack of trust, and more. Ignoring these current barriers puts telehealth at risk for widening disparities in access to care rather than narrowing them. As health systems address these barriers, a focus on provider training will be an essential component. Physicians need to be aware of, address, and apply efforts to mitigate these potential barriers, as well as their own cultural biases and potential personal bias for or against telehealth.

Methods: In September 2020, the AAMC released telehealth competencies for medical students, residents, and practicing physicians. Specific competencies address equity and access, seeking to ensure that providers “understand telehealth delivery that addresses and mitigates cultural biases as well as physician bias for or against telehealth and that accounts for physical and mental disabilities and non‐ health‐related individual and community needs and limitations.” To support academic medicine in the implementation and assessment of these competencies, the AAMC formed a subcommittee of experts to assist in advancing telehealth competency‐based education through scholarship, recognition, and dissemination. Additionally, the AAMC recently awarded programs that demonstrate a positive impact on barriers associated with telehealth, with a specific emphasis on programs training physicians on the health care equity implications of telehealth through the use of AAMC’s telehealth competencies. Through this work, the AAMC is highlighting the need for training in telehealth equity throughout the continuum of UME, GME, and faculty, and seeks to facilitate high quality training by identifying exemplary curricula that adopted the telehealth competencies.

Results: Health systems and medical schools are increasingly focused on addressing health care equity and access through their care delivery and training. To advance health care equity through telehealth and mitigate its barriers, it will be essential to train the current and future provider workforce on telehealth’s impact, both positive and negative, on access and equity. This presentation will include discussions from leaders of medical training programs that are using the AAMC telehealth competencies to train providers on the health care equity implications of telehealth. The presentations will include how the training was designed, how competency is assessed, and how these programs can be adapted or generalized to other settings.

Discussion: With the COVID‐19 pandemic, health systems have implemented or scaled telehealth programs with limited training of most providers and patients. As telehealth modalities increasingly become a routine part of care delivery, it is crucial that providers be educated and trained on its equity and access implications in order to decrease today’s disparities in health care. The AAMC’s telehealth‐ specific competencies, including those on access and equity, are increasingly being used by health systems and medical schools to address this crucial training need. The AAMC aims to identify and spread best practices in training and support health systems and medical schools in their implementation and assessment of competencies to improve the health care of all.

19. Efficacy of a mindfulness‐based programme with and without virtual reality support to reduce stress in university students: A randomized controlled trial

Marta Modrego‐Alarcon,¹ Yolanda Lopez‐del‐Hoyo,¹ Javier García‐Campayo,^1,2 Adrian Perez‐Arand,^1,3,4 Mayte Navarro‐Gil,¹ María Beltran‐Ruiz,¹ Hector Morillo,¹ Irene Delgado‐Suarez,¹ Rebeca Olivan‐Arivalo,³ Jesus Montero‐Marin⁵

¹University of Zaragoza, Health Research Institute of Aragon, ²University Hospital‐Zaragoza, Spain, ³AGORA Research Group, ⁴Autonomous University of Barcelona‐Cerdanyola Del Vallès, ⁵University of Oxford

Background: The mental health of university students is a noteworthy concern. Several studies have acknowledged high levels of stress in this population, with prevalence estimations of 29–37% (Jia & Loo, 2018; Yusoff et al., 2010). University students are continually striving to achieve goals, and their performance is constantly being evaluated (Caballero & Breso, 2015), which generates a great amount of stressors, such as tests, the large amount of content to be learnt, lack of time, getting poor marks, and living up to one’s own expectations (Yusoff et al., 2010). The goal of this RCT was to evaluate the efficacy of a mindfulness‐based programme (MBP) for reducing stress in university students and its action mechanisms and to explore the capacity of virtual reality (VR) exposure to enhance adherence to the intervention.

Methods: The research design was a 6‐week parallel RCT with three arms (‘Mindfulness Based Programme MBP’, ‘MBP + Virtual Reality VR’, and ‘Relaxation’) and assessment periods at baseline, posttreatment, and 6‐month follow‐up. The RCT was conducted following the “Consolidated Standards of Reporting Trials” (CONSORT) guidelines (Schulz et al., 2010). A total of 280 students from two Spanish universities were randomly assigned to ‘MBP’, ‘MBP + VR’, or ‘Relaxation’ (active controls).

The Perceived Stress Scale (PSS; Cohen et al., 1983) was the primary outcome, which is a 10‐item self‐ reported questionnaire in which participants are asked to rate how unpredictable, uncontrollable and overloaded they have found their life over the past month on a 5‐point Likert‐type scale (from 0 = ‘never’ to 4 = ‘very often’). Wellbeing and academic functional outcomes were assessed as well. Multilevel mixed‐effects models were performed to estimate the efficacy of the programme.

Results: Both ‘MBP’ (B = − 2.77, d = − 0.72, p = .006) and ‘MBP + VR’ (B = − 2.44, d = − 0.59, p = .014) were superior to ‘Relaxation’ in improving stress, as well as most of the secondary outcomes, with medium‐ to‐large effects posttreatment and at follow‐up. The long‐term effects of MBPs on stress were mediated by mindfulness and self‐compassion in parallel.

There were significant differences between groups in terms of the number of sessions attended: ‘MBP + VR’ showed more adherence than either ‘MBP’ or ‘Relaxation’ (p<.001). Specifically, the ‘MBP + VR’ group retained significantly more completers than the ‘MBP’ group (p = .005) and the ‘Relaxation’ condition (p<.001).

Discussion: In this RCT, mindfulness training has proven to be more efficacious than relaxation for reducing perceived stress and other psychological and academic‐related outcomes (e.g., anxiety, emotion regulation, academic engagement, and burnout) in a sample of Spanish university students post intervention and at the 6‐month follow‐up, with moderate to‐large effect sizes.

The inclusion of VR exposure resulted in an improved adherence to the mindfulness programme, although it did not affect the efficacy of the intervention.

The present work also observed that mindfulness and self‐compassion were significant mediators of the long‐term effects of MBPs on perceived stress.

Other academic institutions of higher education are encouraged to introduce different initiatives, such as this one, to help care for students’ mental health.

20. Multicenter Study Assessing Physicians’ and Transport Teams’ Attitudes and Expectations about Utilizing Telemedicine to Manage Critical Neonatal Transports.

Tavleen Sandhu MD,¹ Jawahar Jagarapu MD,² Rashmin C. Savani MD,² John Chuo MD,³ Abeer Azzuqa MD,³ Lise DeShea PhD,¹ Abhishek Makkar MD¹

¹The University of Oklahoma Health Sciences Center ²The University of Texas Southwestern Medical Center/ Children’s Medical Center ³Children’s Hospital of Philadelphia ⁴University of Pittsburgh School of Medicine/UPMC Children’s Hospital of Pittsburgh

Background: Live video‐conferencing via mobile devices and computers has opened channels for further advancements in telemedicine that may provide indispensable services to patients in remote areas. Managing critically ill neonates has unique challenges, and the transport team plays an important role in stabilizing and facilitating the transfer of these neonates from lower level nurseries. We conducted a multi‐center study to assess the readiness for utilizing telemedicine as an adjunct to guide the care of critically ill neonates amongst physicians and transport team members (TTMs). This is the first multi‐center study that explored physicians’ and TTMs’ perceptions of telemedicine usage and its value in neonatal transport. The findings in the study might help explain the human factors that could influence adoption and implementation of tele‐transport programs.

Methods: An anonymous, voluntary survey on pre‐implementation attitudes toward telemedicine usage during neonatal transport was conducted as part of a quality improvement initiative. This survey involved physicians and TTMs from four institutions whose responses were entered into an online survey using REDCap®. The survey inquired about satisfaction with the current practice of phone consultation and the perception of using telemedicine to optimize the management of neonates during transport.

Results: The overall response rate for the survey was 78.1% for physicians and 54.7% for TTMs. Half of the 79 physicians and less than one‐fourth of the 64 TTMs had prior experience with different application of telemedicine other than that used on neonatal transport. TTMs expressed greater concern about the inconvenience of video (54.7% vs. physicians 35.4% agree or strongly agree) and its time‐consumption (84.4% vs. physicians 50%). More than 70% of physicians and less than half of TTMs endorsed the potential for added value and quality improvement with video capability. 49.2% of TTMs reported concern about video calls reducing their autonomy in patient care. Physicians expressed confidence in management decisions they would make after video calls (72% confident or very confident) and less confidence (45.1%) about both the phone assessment by TTMs and their decisions based on phoned assessment. In contrast, TTMs were confident or very confident (93.8%) in both sharing their assessment over the phone and executing patient management after a phone call, compared with 68% for decisions made after video calls.

Discussion: Physicians and TTMs had distinct opinions on the potential effects of telemedicine during neonatal transport. Physicians were more likely than TTMs to agree with statements about the potential for improving quality of care, while TTMs were more likely than physicians to say video calls would be time‐consuming and inconvenient. We speculate some differences may stem from the TTMs’ concern of losing their autonomy. Therefore, during implementation, it is critical for physicians and TTMs to agree on a shared mental model of indications for telemedicine during transport and its value to the patient care. More empirical data is needed in this area.

21. Video‐Teleconferencing for Disease Prevention, Diagnosis, and Treatment: a Rapid Review of the Evidence

Jordan Albritton PhD, Alexa Ortiz MSN, Candi Wines MPH, Graham Booth BS, Michael DiBello MPH, Stephen Brown MS, Gerald Gartlehner MD, Karen Crotty PhD

RTI International, Insight Policy Research

Background: The use of video‐teleconferencing (VTC) as a substitute for in‐person health care or as an adjunct to usual care has increased dramatically in recent years. The COVID‐19 public health event (PHE) has further driven a dramatic increase in the utilization of VTC, with nearly three times as many patients reporting ever using video visits in 2020 compared with 2019. Although some policy changes enacted during the PHE are likely to be made permanent, the debate continues regarding the use and reimbursement of VTC in healthcare and there remains considerable uncertainty around the relative benefits and harms of VTC. The objective of this rapid review was to systematically assess recent evidence of the effectiveness and harms associated with VTC visits for disease prevention, diagnosis, and treatment and to produce an evidence map summarizing the current state of the evidence including notable gaps.

Methods: We followed guidance from the Cochrane Rapid Reviews Methods Group and international Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) reporting guidelines. To identify relevant literature, we systematically searched PubMed, Embase, Web of Science, and the Cochrane Library from January 1, 2013, to March 3, 2021. We included all RCTs investigating the use of VTC for disease diagnosis, prevention, or treatment compared with a broad definition of usual care. We excluded mental health and substance use disorders, maternal care, and weight management, as well as studies from countries that were not rated very high on the Human Development Index and studies with sample sizes less than 50. We searched for a wide range of outcome types, including clinical effectiveness outcomes, harms, healthcare utilization, attitudes and experience, and process measures. We rated risk of bias for all included studies and abstracted data from studies rated as low risk of bias or having some concerns of bias. We created an evidence map and narratively synthesized the evidence.

Results: We identified 643 unique records, of which 43 publications representing 38 primary RCTs met our inclusion criteria. Of these, 20 RCTs were rated low risk or of some concerns of bias and were included in the data synthesis. The most commonly studied conditions included diabetes (n = 4), respiratory conditions (n = 4), pain‐related disorders (n = 4), and heart failure (n = 3). Thirteen of these 20 studies focused on multicomponent VTC interventions. Overall, VTC for the treatment and management of disease produced largely similar outcomes compared with the control group (e.g., in‐person care, audio‐only telehealth, or no care). Fourteen studies reported no statistically significant differences between groups. Six studies, five of which involved multicomponent interventions, reported primary clinical, healthcare utilization, or patient satisfaction outcomes that favored the VTC intervention. Six studies included subgroup analyses and largely found no differences in the effect of VTC based on age, gender, or education. Two studies reported mixed findings with respect to difference between rural and urban patients. No studies evaluated the impact of VTC on health equity or disparities, and none studied the diagnosis or prevention of disease.

Discussion: Findings from this rapid review found that replacing or augmenting aspects of usual care with VTC results in similar clinical effectiveness, healthcare utilization, patient satisfaction, and quality of life. However, included studies were limited to a handful of disease categories. Some evidence suggests that multicomponent VTC interventions may offer improvements in outcomes over usual care, but additional work is needed to determine which combinations of multicomponent VTC interventions result in improvements and for which diseases and subgroups of patients. There are several gaps in the current evidence base including the effectiveness and harms of using VTC in underserved and vulnerable populations especially in relation to health disparities and health equity. Also lacking, is any evidence related to the use of VTC for collaborative or integrated care, and use of VTC for patients with multiple chronic conditions.

22. Knowledge, attitudes, and practices of telemedicine education and training of medical students from 10th largest medical school in the U.S.

Madison Karakash OMS‐II, Emily Gauthier OMS‐II, Assad Ali OMS‐III, Marie Florent‐Carre DO, Deborah Mulligan MD

Nova Southeastern University Kiran C. Patel College of Osteopathic Medicine

Background: Over the past few decades as technology has progressed, telehealth has grown exponentially in the United States. The COVID‐19 pandemic has increased utilization of telemedicine and served to fuel continued growth into the future. The rapidly accelerated move toward telemedicine has provided an opportunity for medical schools to guide their students in developing the competencies for this transition. By integrating training into existing curricular structures, medical schools can expose students to telemedicine without significant additional burden. At one accredited osteopathic medical school, a ten‐hour telemedicine module was developed and integrated into an online required course to provide virtual training modules during standard medical school education. Completing telemedicine training during medical school will provide medical students the foundation to build competency in skills valuable in improving the quality, access, and cost of care for future patients.

Methods: Third‐ and fourth‐year medical students (n = 410) from an accredited osteopathic medical school were enrolled in a required telemedicine course within the Department of Rural and Urban Underserved Medicine. Within this course was a pre‐course survey, ten on‐line modules on telemedicine, mock virtual healthcare delivery assignments, and ultimately a post‐course survey. Each student completed the pre‐course survey prior to exposure to the telemedicine training modules, and then completed the post‐course survey at the conclusion of the course. The surveys were created to outline baseline medical student perspectives on telemedicine, as well as document the experiences and learnings that resulted from completing telemedicine coursework during standard medical education. As part of the final course evaluation, a recorded mock telemedicine visit was required to demonstrate proficiency in telemedicine skills prior to completion of respective third‐ or fourth‐ year of medical school. Based on this data, an assessment of the capacity to integrate telemedicine learning into current medical education is provided. Responses were recorded on Canvas (a nationally used resource tool for universities) throughout the duration of the course.

Results: The pre‐ and post‐survey results demonstrate support of telemedicine medicine in practice and the benefit of integrating telemedicine education into the current curriculum. Among third‐ and fourth‐ year medical students, ninety‐four percent (94%) agree telemedicine is a valuable service to offer patients while seventy‐four percent (74%) of third‐year and seventy‐eight percent (78%) of fourth‐ year students agree physicians can provide quality care to their patients via telemedicine. In regards to the course, eighty‐five percent (85%) of third‐year and sixty‐one percent (61%) of fourth‐year students agree telemedicine is a valuable component to their clinical education. Following the post‐ course survey, seventy‐eight percent (78%) of third‐year and sixty‐nine percent (69%) of fourth‐year students stated the course enhanced their knowledge of telemedicine while sixty‐five percent (65%) of third‐year and seventy‐two percent (72%) of fourth‐year students stated the use of telemedicine will improve their career performance in professional practice.

Discussion: The results from pre and post course surveys depict a favorable platform for the integration of telemedicine modules into medical school curriculums. Medical students are supportive of the additional coursework and understand the need to enhance their telemedicine skills to provide virtual care. Healthcare apps and digital platforms are ubiquitous. Through the incorporation of a telemedicine course with traditional medical education, medical schools can create a platform that is both easily adapted and utilized by medical students. The development of technological skills in today’s medical student, combined with a proficiency in traditional office‐based medicine, may pave the way for more accessible, effective patient encounters.

23. Providing on‐site mental health crisis service via telehealth to aid law enforcement in rural South Dakota

Kimberly A. S. Merchant MA, Priyanka Vakkalanka PhD, Molly Johnson, Megan Wexler, Mark Pals, Marcia M. Ward PhD

University of Iowa, Avel eCare, LLC

Background: With a grant from The Leona M. and Harry B. Helmsley Charitable Trust, Avel eCare, LLC offered a tele‐behavioral health service called Virtual Crisis Care (VCC) to county Sheriff’s departments across South Dakota. The 24‐hour service provides law enforcement officers with technology on tablets to establish a virtual video connection with mental health professionals located in the Avel eCare, LLC hub in Sioux Falls. It is often used to help deescalate crisis situations, stabilize and assess the safety of the individual, and determine a plan of care. Avel eCare, LLC clinicians provide recommendations to law enforcement officers after the encounter.

VCC’s pilot program was initiated in the field in January 2020. Through fall 2020, 18 Sheriff’s departments and a Judicial Circuit Court Probation Office, representing eight counties’ probation offices, joined the program.

Methods: A research center at the University of Iowa was contracted to do an evaluation of the pilot project. The VCC evaluation consisted of quantitative and qualitative components. The quantitative data was collected by Avel eCare, LLC in its VitelNet information system. Data available included frequency of use, location of use, gender of person served, the nature of the request for service, outcome of encounter, and disposition of person served. The qualitative component included 14 interviews with 15 law enforcement officers in 10 counties. The UI research team conducted interviews from January through May 2021. Five counties had multiple representatives interviewed or included in interviews with one county extending interviews to city police department users. These 10 counties experienced 155 of the 181 VCC encounters completed in South Dakota over the 18‐month pilot. Using a semi‐ structured interview guide, law enforcement officers were asked questions that included, but were not limited to why their county was interested in VCC, motivation to use the service, descriptions of typical as well as unique situations when used, community reactions and willingness to use, barriers to use, overall opinion of service.

Results: Law enforcement officers responded to calls – most coming from family or neighbors who have concerns for friends or loved ones ‐ that presented requests for crisis intervention where consultations with VCC for a variety of mental health situations. These included self‐harm and suicidal ideation (53%), anxiety and depression (27%), aggressive/disruptive behavior and bullying (16%), and delusional behavior (4%). VCC was used mostly by male persons in crisis (62%). The most common locations for using the service were in the community (45%) or in the home (40%), although it was used in jails as well (15%). Law enforcement officers who used VCC described many benefits including saving time and costs, ease of operation, increased officer confidence, providing community support that is readily accepted, having an option other than jail or involuntary commitment, and having complete provider documentation and recommendations. Many officers said the service was extremely valuable, talked about expanding to schools, and wanted the service to be more widely available across rural South Dakota.

Discussion: There are a variety of examples where mental health counselors accompany law enforcement officers on mental health crisis calls. Having a mental health professional available to assist has proven valuable for law enforcement and the community. However, having a person available in rural areas is a challenge. Therefore, using telehealth to bring a mental health professional to the crisis encounter works to help people in rural and underserved communities by offering timely mental health services. Evaluation indicated that VCC provided that needed service. The pilot program was funded by a generous grant. Noting the acceptance and benefit to rural counties, The Helmsley Charitable Trust, Avel eCare LLC, and Criminal Justice Initiative formed a partnership to reach out to South Dakota legislators to promote the service statewide to all 66 counties. In the 2021 session, South Dakota legislature provided appropriate funding to sustain VCC in existing sites and expand to additional counties.

25. Use of telemedicine for children in foster care during the COVID‐19 pandemic

Kristine Fortin MD, Judith Dawson RN, Philip V. Scribano DO,

Children’s Hospital of Philadelphia, University of Pennsylvania

Background: There is a high prevalence of complex health care needs and access barriers among children in foster care. Frequent placement changes and consenting complexities are common barriers to care that are unique to children in foster care. Given the special health care needs of children in foster care, focused studies of healthcare delivery to this population are warranted. There have been many reports of telemedicine programs deployed during the COVD‐19 emergency. These studies are also important to inform optimal and equitable telemedicine use post‐pandemic. To our knowledge, prior studies of telemedicine programs deployed during the pandemic have not focused on children in foster care. Our objectives were: 1. to describe outcomes of a telemedicine program serving children in foster care during the pandemic; and 2. to compare outcomes between telemedicine and in‐person comprehensive health assessments (CHA).

Methods: Our specialty clinic for children in foster care conducted CHA via telemedicine when in‐person visits were restricted during the pandemic. Our institution conducts telemedicine through our patient portal app. Consent is required during the telemedicine login process, a barrier for children in foster care who often present with a caregiver who is not authorized to consent. Advocacy efforts led to our county’s commissioner of child protective services providing a letter consenting to telemedicine visits for children in care during the pandemic. A nurse care coordinator assisted caregivers with the app, as needed. Outcomes of telemedicine referrals were tracked in REDCap. Questions from the validated Telehealth Usability Questionnaire (TUQ) were completed by fellow and attending physicians after each visit. Physicians rated their ability to hear, express themselves, and visual quality on a scale of 1 (strongly disagree) to 5 (strongly agree). Prevalence of completed lab tests, medication prescriptions, and referrals to health care services were compared between patients undergoing telemedicine CHA and patients undergoing in‐person CHA the year prior at the same clinic. Descriptive statistics, Chi‐2 and Fischer‐exact tests were used.

Results: Of 91 patient referrals, 83 (91%) resulted in completed telemedicine CHA; 3 never logged in to scheduled visit(s); 3 had prohibitive technology barriers; and, 2 were determined to require in‐person CHA. Mean age of telemedicine patients was 9.2 years (SD 5.7). Attendance rate was 83/92 (90%) as 7 patients missed 9 appointments. Mean duration of visits was 66 minutes (SD 24.5). Response rate for TUQ (81 sent to attendings, 58 to fellows) was 98%. Mode response for ability to hear patients and express self was 4 (agree), while mode for visual quality was 2 (disagree). Of patients seen by telemedicine, 27% received medication prescriptions, and 77% were referred to at least one health care service in addition to primary care. While 57% of patients seen in‐person underwent laboratory tests, only 1% of telemedicine patients got testing (p<.001). Telemedicine patients were significantly less likely to be referred for vision services compared to patients seen in‐person (p = .03). Compared to in‐person patients, telemedicine patients received less referrals for medical specialties (65% versus 41%, p<.001) and less prescriptions for new medications (26% versus 13%, p = .02).

Discussion: When in‐person visits were restricted during the pandemic peak, telemedicine allowed for healthcare delivery to children in foster care. Over 2/3 of telemedicine patients received referrals and <1/4 received prescriptions. High visit completion rates suggest that telemedicine was accessible to most families. Important components of CHA include physical exam, vision screens and lab work.

Significantly lower rates of lab work and vision referrals in telemedicine compared to in‐person patients suggest need to complete these components in‐person. Also, low physician ratings of visual quality suggest limitations of visual exam. These findings could inform post‐pandemic use of telemedicine. Once components of CHA such as detailed exam and lab work are completed in‐person, telemedicine could be an effective tool for follow‐up of ongoing needs. Continued advocacy to overcome unique consent issues and potential technology barriers is needed for this population with special health care needs.

26. Using Remote Patient Monitoring to Wean Patients from Feeding Support

Micah Dean MBA, Kylee Denker RN, Courtney Sump MD, Stephanie Oliveira MD

Cincinnati Children’s Hospital

Background: The Telehealth Program and Division of Gastroenterology at Cincinnati Children’s Hospital Medical Center (CCHMC) designed and launched a novel remote patient monitoring (RPM) program for patients receiving feeding support via total parenteral nutrition (TPN), NG tubes and G‐tubes. The goals of the program are to reduce weaning time from feeding support, decrease days away from home and improve patient and family experience.

Methods: Patients are enrolled in the program prior to hospital discharge. Application based RPM technology is used to track patients’ input and output, weight measurements, medications and general health trends. Qualitative data is submitted daily and weights are submitted twice per week by family members through an app. The patient reported data is reviewed by a centralized team of RPM nurses and is used by the clinical team to make adjustments to TPN amounts and IV fluid volumes rather than waiting until the next scheduled clinic visit. Once patients are tolerating full enteral feeds with stable output and achieve a healthy target weight, TPN is discontinued and a patient may graduate from the RPM program.

Results: Patient enrollment began in March 2021 and is ongoing. Sixteen patients ranging 4 months to 3 years old have been enrolled to date. Two patients have either never reported data or discontinued participation. Fourteen of sixteen patients (88%) of patient enrolled participated for at least two weeks. Program adherence for this patient population is 78.9%. Adherence is defined as the submission of all requested health data and vital signs on a given day. In addition to the clinical outcomes, patient experience data is also collected to determine families’ feelings about providing health information through an app to improve care. Responses are positive with most families highlighting the ease of communication (HIPAA compliant messaging, video visits and direct phone lines) available within the RPM app. 100% of respondents (N = 12) indicated they would recommend RPM to a friend or family if they had a qualifying condition.

Discussion: Traditional care models for patients receiving TPN therapy require regularly scheduled clinic visits to assess the tolerance of enteral feeds and output measurements in the setting of weight loss/gain and appropriate adjustments to the treatment plan. This may result in unnecessary delays in transitioning off of TPN while waiting for clinic visits to make changes to the care plan. Though we are still early in our journey, we hope to see positive results for TPN patients enrolled in RPM, including faster weaning times and decreased time to achieving a target weight. Additional programs to monitor feeding support have been launched in failure to thrive and g‐tube population and have enrolled 59 patients.

27. “The doctor on the laptop”: Adaptation of Inpatient Psychiatry Fellowship Training to Telehealth

Danielle Wentzel DO, Mitchell Douglass MD, Sharon E. Cain MD, Stephanie E. Punt MA, Ilana J. Engel MA, Annaleis K. Giovanetti MA, Madeleine Hardt MA, Eve‐Lynn Nelson PhD

Children’s Mercy, University of Kansas Medical Center, University of Missouri‐Kansas City

Background: Telepsychiatry improves access to care and increases trainee exposure to child and adolescent populations not typically encountered. Research related to telepsychiatry’s use in medical education remains sparse. Using educational case examples representing the breadth of vulnerable patients served, we aim to demonstrate our process for and experience with adapting clinic workflow and expanding clinical experiences to medical trainees in order to meet specific educational goals of the fellow via inpatient telepsychiatry.

Methods: The fellow reviewed cases from her inpatient telepsychiatry rotation in order to highlight cases that exemplify use of telehealth in this unique setting and how these experiences map onto Accreditation Council for Graduate Medical Education (ACGME) milestones. Cases were chosen that demonstrated her diverse telepsychiatry training with a heterogeneous patient population in order to showcase the specific strengths and challenges of telepsychiatry in this context. These cases help to illustrate how training goals and patient care were adapted for telepsychiatry. Equivalency in meeting the fellow’s training goals and patient care needs was assessed.

Results: This inpatient telepsychiatry rotation model, developed quickly in response to the restrictions of the pandemic, allowed the child psychiatry fellow the educational experience that encompassed ACGME milestones of providing care to acutely ill children in the inpatient setting. The use of telepsychiatry limited exposure to COVID‐19 for the fellow in the inpatient setting and did not impede the educational experience. Interview skills were developed extensively via telepsychiatry. This model was also successfully implemented with vulnerable populations, such as those with psychosis, developmental delay, and complex medical needs.

Discussion: The adaptation of this rotation kept trainees safe during the COVID‐19 pandemic and could continue to serve a valuable purpose outside of the pandemic by providing a unique platform through which trainees with health conditions, such as pregnant trainees on bedrest, are still able to make positive movement toward training goals while working from home. These expanded options could increase workforce diversity and prevent workforce gaps. This model of inpatient telepsychiatry clinical interviews could serve as a tool to allow more students and trainees to witness unique cases as patients were able to tolerate more student observers in the online format. Clinically, greater effort to incorporate the fellow virtually into the multidisciplinary treatment team would be beneficial. Future research could identify additional ways to improve job equity.

28. Improving Access to Behavioral Health Services Using Telemedicine in Rural American Indian Communities

Stephanie Yang BA, Sarah Haynes PhD, James P. Marcin MD, Murat Pakyurek MD, Bibiana Restrepo MD, Jeffrey S Hoch PhD, Peter Yellowlees MD

UC Davis Health

Background: Mental health services are urgently needed in rural America, especially for American Indians populations. Geographic isolation and shortage of behavioral health professionals present challenges to accessing behavioral health services in rural areas. To help fill this gap in Northern California, we developed the Tele‐Behavioral Health for American Indians Affected by Mental Illness (Tele‐AIMI) program. The goal of TELE‐AIMI is to increase access to behavioral health services using telemedicine in American Indian communities. We partnered with four Indian Health Services (IHS) located in rural northern California to implement the program, which includes tele‐behavioral health consultations with a UC Davis psychiatrist during a patient’s in‐person visit to their primary care provider (PCP).

Here we summarize our program structure, the patient population served by our program, and lessons learned to inform future telehealth programs targeting rural and underserved populations.

Methods: The UC Davis Center for Health and Technology works with each participating IHS site to schedule a consultation with a UC Davis psychiatrist after receiving a referral from the PCP. The patient attends the visit at their local clinic site, where they have a private consultation with the psychiatrist over telehealth. At the end of the consultation, the PCP joins the visit and works with the psychiatrist and the patient to develop a patient‐centered treatment plan that may include medication, referrals to counseling or other services, and a plan for follow‐up. For each tele‐behavioral health consultation, we administer surveys to collect demographic data and measure patient‐reported outcomes including depression, anxiety, overall health, physical function, and substance abuse using five validated clinician‐rated instruments (PHQ‐9, GAD‐7, PROMIS Global‐10, PROMIS‐Physical Function, DUDIT‐C). In addition, we also use two provider‐observed validated instruments (CGI and GAF) to measure illness severity and functioning, and collected ICD‐10 codes to determine primary diagnoses.

Results: As of July 2021, we have provided a total of 133 tele‐behavioral health consultations with a psychiatrist through the Tele‐AIMI program at four participating rural IHS clinics. The most common primary diagnoses were anxiety (25.6%), depression and mood disorders (23.3%), schizophrenia and other psychotic disorders (13.5%), and bipolar disorder (9.8%). Most patients (63.9%) were female and the median age was 39. Of those who reported insurance status, only 6% of patients had private insurance; the remaining 94% were insured by Medicaid or Medicare. A total of 35 patients completed the surveys; the other patients refused or did not have time to complete the surveys at their appointment. Patient‐reported outcomes revealed high levels of anxiety and depression, with 38.9% reporting severe anxiety and 37.1% reporting moderately severe or severe depression. Having both a clinician and staff champion at the local site improved the chances of successful scheduling and completion of a tele‐behavioral health consultation. Scheduling and no‐shows are two main challenges facing the program.

Discussion: Our program demonstrates a model of care that can be used to successfully increase access to behavioral health services for rural and underserved populations. Our program provided behavioral health services that may have taken a long time to schedule (resulting in delays in care) and would likely require long travel distances for patients. This type of telehealth program may also improve patients’ willingness to receive behavioral health services by allowing patients to receive these services in their medical home and with the collaboration of a familiar provider. Furthermore, this type of telehealth program improves care coordination by allowing behavioral health providers to communicate with both the patient and the PCP in real time to develop a treatment plan that is acceptable to all, with specific roles outlined for each person. Survey completion was low among patients in our program; providing online options for survey completion may improve survey rates.

29. Designing a telehealth system to assist patients with navigating the continuum of care between urgent care and primary care

Ken Tegtmeyer MD, Victoria Ames MPA

Cincinnati Children’s

Background: With high patient satisfaction, direct to consumer urgent care telehealth services are growing in utilization. These services can create tension between those offering the service and community providers managing the medical home. Community providers voice concerns over how these services are communicated to patients and lack of coordination with the patient center medical home, creating confusion for patients on the continuum of care. Little information is available discussing coordination between telehealth urgent care services and the patient centered medical home.

Methods: We worked with General Pediatrics Primary Care and Emergency Medicine to jointly design a system to assist patients with navigation between two telehealth offerings. The design created a new On Demand Primary Care Nurse Triage Telehealth service (Triage) on the same telehealth platform as an existing On Demand Direct to Consumer Urgent Care service (DTC). An eligibility file uploaded weekly into the telehealth platform allows the system to provide access to Triage during regular working hours, and extended access to DTC in off hours. All visits require video. Patient communications were created for General Pediatrics Primary Care patients. We also obtained data from both the vendor portal as well as our own electronic medical records system to support the process.

Results: From April 20, 2021 through July 31, 2021, there were 34 Triage visits. There were a total of 455 DTC telehealth visits during this time. The top three reasons for a Triage visit were Skin concerns, Breathing or asthma concerns, and Cough, cold or congestion. The top three reasons for DTC visit were Skin concerns, Cough, cold or congestion, and Eye concerns.

Discussion: Navigating the healthcare continuum is not easy for patients. Designing a system that relies on technology to help patients navigate care can bridge the gap between the patient centered medical home and urgent care telehealth services. With this design, there were likely 34 visits appropriately navigated to the patient centered medical home instead of the DTC service. Supporting this continuum of care approach leads to an improved experience for patients and community providers, alleviating confusion and tension, and truly providing patients with the right care at the right time.

30. Addressing disparities in pediatric telehealth access

Kori Morrison NA, Samuel P. Hanke MD, Michael C. Ponti‐Zins, Jennifer Ruschman, Ken Tegtmeyer MD

Cincinnati Children’s

Background: There was a surge of telehealth across the country and the globe during the COVID19 pandemic. The impact of this significant change in healthcare delivery is largely unknown. Improved patient and family experience (PFE) is an established tenet of the triple aim of healthcare (Berwick et al 2008). Formal evaluation of PFE was limited in telehealth prior to COVID‐19. A national report from the Agency for Healthcare Research and Quality identified that access to care improvements for Latino and Black Americans were disproportionally experienced. During the COVID‐19 pandemic, telehealth became the primarily mechanism for access to healthcare at our organization. We first sought to understand, the potential impact of implementing a new technology‐based method of healthcare across different demographic and social determinates of health. Next, we utilized the results to identify gaps in PFE and access and implement interventions aimed to reduce disparities in access.

Methods: Aim 1: PFE survey data was collected from March 15, 2020 through November 30, 2020 using the NRC Real‐time survey platform following all ambulatory telehealth encounters. We evaluated the overall rating of the telehealth service on a 10‐point likert scale. Promoter experience was defined as 9 or 10 out of 10. Additionally, we evaluated the experience with the telehealth technology on a 4‐point Likert scale. Ideal experience was defined as a percent of top response. Responses were then analyzed by patient reported demographics in the EHR. Ambulatory in‐person utilization was determined for the 8 months prior to COVID‐19 pandemic (July 2019‐March 2020). Predicted telehealth utilization was estimated and compared to observed telehealth utilization for the first 8 months of the COVID‐19 pandemic (March 2020‐November 2020).

Aim 2: A gap in access to telehealth was identified for Spanish speaking families. We then developed a process map, looking at experience prior to the telehealth visit and identified gaps in the communication channels and materials used to inform families how to access telehealth.

Opportunities to improve the processes and content for Spanish language families were specified and prioritized.

Results: The overall experience with telehealth was favorable with 84.8% promotors (8695 total surveys). Response rate for all encounters was 15.3% with no differences in response by demographic. Spanish speaking families were utilizing telehealth visits 49% less than predicted volumes from previous in person visits. Hispanic families were utilizing telehealth 25% less than expected. The overall rating of the service for the telehealth visit was similar for Spanish speaking families as compared to English speaking families (90% promoter Spanish: 85% promoter for English speaking families). When asked about if the method of connecting to the visit was easy, there was a large difference where 73% of English‐speaking families indicated “Yes, definitely” but only 56% of Spanish speaking families made that same choice. Several process changes were implemented to improve the experience and utilization for telehealth for Spanish families. These included:

• Materials were created in Spanish to support understanding what are video visits

• Website was updated with text and videos in Spanish on instructions for how to connect

• Reminder texts were created in Spanish and we immediately connect a Spanish interpreter for technical support

Discussion: Adding technology into our healthcare delivery models can add another layer of complexity that could further those inequities. The intent of telehealth is to make healthcare more accessible to families, and to minimize the inequities and access challenges. Telehealth can be an excellent alternative to in‐ person pediatric care providing a high overall experience rating across different demographic characteristics. Despite these similarities in the overall experience with telehealth, we did find differences in the experience with telehealth technology greatest for our Spanish speaking and Hispanic families. We were able to create content to address inequities. Failure to understand the different experiences and access may unintentionally contribute to the digital divide. By allowing patient and family data to inform our approach, we seek to make our telehealth experience not only more equitable, but also more accessible to the Spanish speaking families we serve.

31. Using Technology for Social Wellness During COVID‐19 in Older Adults

Jared Santiago, Cynthia White‐Williams PhD

University of North Florida

Background: The COVID‐19 pandemic has ushered in unprecedented challenges to mental and social well‐being. To mitigate the spread of COVID‐19, the Center for Disease Control established safety guidelines to promote physical distancing. Prior to the pandemic, a significant portion of older adults was considered socially isolated; the stay‐at‐home orders increase the risk of social isolation. While anxiety and fear are normal responses to adversity, for those already pre‐disposed to isolation and loneliness, the potential for exacerbation of these issues can lead to challenges in social well‐being. However, technology can support social connectedness and physical distancing. Electronic features such as texting, calling, and video chatting assist in communication and social interaction during the pandemic. The purpose of this study was to investigate how COVID‐19 impacted social isolation and loneliness among older adults, and the perception and use of technology to enhance social connectedness.

Methods: The study was conducted with the approval of the Institution’s Review Board and in partnership with two Continuing Care Retirement Communities in Northeast Florida. We surveyed Older Adults who live in assisted living and independent living facilities. We used SPSS version 25 to describe the study population and Chi‐squared to examine study objectives. We used Qualtrics to deliver a survey that includes the UCLA Loneliness Scale Short Form. We also included questions to analyze technology utilization before and during the pandemic and gather perceptions on technology. The survey was available from March 2021 to May 2021.

Results: One hundred twenty‐one persons completed the survey and were included in the analysis. In the sample, the average age was 84 years, 66.9% were female, 95% were White, 63.6% were widowed, and 34.7% reported at least a bachelor’s degree level of education. Chi‐squared was used to examine changes in technology use before and during the pandemic. Results were statistically significant for an increase in the use of video chat (p‐value 0.02), social media (p‐value 0.046), phone (p‐value 0.029), and text messaging (p‐value 0.00). 40% report feeling a lack of companionship (10% often, 30% sometimes); 32% reported feeling isolated (12% often, 20% sometimes); and 41% reported infrequent social contact (17% often, 24% sometimes). Technology utilization for social connectedness was assessed through a series of ten questions: 22.7% were favorable, 36% were neutral, and 37% were unfavorable towards technology utilization. However, 49% agreed that online communication did not contribute to loneliness and 45% preferred phone communication over other forms of technological communication.

Discussion: The results suggest that concerns about isolation and loneliness in older adults are well‐founded. During COVID‐19, older adults used alternative means to stay connected with their social network, reporting a significant increase in technology use during the pandemic with a strong preference for telephone communication. This evidence suggests that, during a public health crisis, technology is a viable solution with a preference for simpler forms of technology‐mediated communication. As expected, the perception of technology was doubtful, but the results suggest that older adults recognize technology to maintain social connectedness and mitigate loneliness. Assisted living and independent living administrators should consider efforts to intentionally provide technological support and encourage interaction when physical contact is not in the interest of public health.

32. Haunted House: The Dangers and Ghosts of the Lived Environment

Kathryn Neill PharmD,¹ Duston Morris PhD,² Debbie Knight PharmD,³ Pamela de Gravelles PhD,¹ Angel Holland DPT,¹ Wendy Ward PhD,¹ Karen Dickinson MD¹

¹University of Arkansas for Medical Sciences ²University of Central Arkansas ³Harding University

Background: The Arkansas Interprofessional Education Consortium (ARIPEC) is a statewide collaborative with members from University of Arkansas for Medical Sciences (LR and NW campuses), University of Central Arkansas, Harding University, University of Arkansas – Fayetteville, University of Arkansas – Little Rock, Arkansas State University ‐ NYITCOM, and Pulaski Technical College. This group works to develop and deliver relevant, meaningful interprofessional learning experiences for students from more than 70 different professional training programs. In March of 2020, COVID‐19 arrived in the state and challenged ARIPEC to maintain delivery of impactful interprofessional activities while minimizing COVID‐19 risk. One joint venture was the creation of a novel virtual home assessment simulation.

Methods: A unique simulated environment case was developed as a virtual adaptation for interprofessional team clinical case encounter in AY2020‐21. Faculty from three universities created rooms illustrating patient characteristics, hazards, habits, and interpersonal considerations (older couple, teenager and young child). Each university mocked up and video recorded one room (kitchen, bedroom, and living room) which can be used stand‐alone or combined to represent a home.

Over 4 sessions students (n = 400; medical, pharmacy, physician assistant, dental hygiene, communication science disorders, physical and occupational therapy, addiction studies, respiratory care, radiography, public health, sonography, and nursing) evaluated the recorded environments using the INHOMES assessment tool. Students created action plans for immediate needs and for when weight bearing status allowed increased mobility and plans identified professionals required to meet these needs. A pre‐/post‐questionnaire included the Interprofessional Collaborative Competency Attainment Survey (ICCAS). A 5‐point Likert scale evaluated questions for simulation methodology, home assessment, and overall impression. Mean scores are reported.

Results: All ICCAS metrics increased pre‐ to post‐evaluation. Students indicated the experience helped gain confidence (4.32), developed communication skills (4.34), reasoning skills (4.41), and decision‐making skills (4.41). Debriefing was helpful for professional development (4.46). Students felt more comfortable in the ability to complete a home assessment to identify safety hazards and concerns (4.48) and identify team members to meet the immediate and long‐term needs for a patient with pain and limited mobility (4.48). Students indicated the activity demonstrated the value of providing team‐based home assessment education (4.54) and overall was a valuable education activity (4.48). Students felt the recorded rooms portrayed the simulated environment well (4.5), gave constructive indicators to identify patient characteristics and behaviors (4.53), and provided an effective mechanism to learn home assessment using the INHOMES tool (4.52).

Discussion: Review of a video recorded simulated home environment was successful in supporting development of an interprofessional action plan for a home assessment using the INHOMES assessment tool. During debriefing, students cited the utility of the digital platform and recording as a novel way to gain information about a patient’s day‐to‐day habits and behaviors and partner with other professionals to provide specific patient‐centered recommendations based on the team’s observations. In particular, students noted this method would be a meaningful tool for patients in rural areas or those who don’t have access to home health visits. Students also noted that it is rare to have a patient that does not have a smart phone, so a patient could complete his/her/their own recording to submit to the care team. We recognize significant opportunity to expand care using this digital health/telemedicine format.

33. Connecting Two Lungs: Implementaiton of Telehealth Technology to meet Pre‐Transplant Patient Education needs

Daniel Miller MHA, Liz Deleener MBA, Nancy P. Blumenthal DNP, Kate Ventura MSN

Hospital of the University of Pennsylvania

Background: Patient and caregivers are required to undergo organ specific transplant education during their evaluation for transplant candidacy. In this single center experience, patients and support people are required to attend onsite education before placement on the transplant waiting list. Transplant nurse coordinators have identified logistical challenges to meeting this requirement often result in patient dissatisfaction and delayed time to wait listing. HIPPA compliant technology has been leveraged to fill this gap in care by delivering patient education in a video conferencing format. Through telehealth activities, our organization has pursued performance improvement strategies focused on increasing accessibility for patient and caregivers by reducing travel burden for those coming from afar.

Methods: Professional guidance was provided by institutional Connected Health services. With their assistance connected health, innovative telehealth strategies using audio video conferencing software (Vidyo) was utilized to connect distant learners (learner) to transplant nurse coordinator (educator) through live virtual support education class that aimed to supplement our current patient education model. The scope of the project initially limited to pre‐transplant ambulatory evaluation patients.

Participants were invited to participate in the online education through email invitation. Classes were led by a nurse coordinator in the evening once a month in attempt to attract those who have conflicts during standard operating business hours. Content of the class focused on relevant transplant topics such as evaluation testing, transplant wait list, donor criteria, and post‐transplant care. Afterwards, post education surveys were emailed to the participants for the intent of measuring the level of satisfaction with method delivery and accessibility.

Results: The virtual telehealth education class pilot took place over 22 months and has educated 124 total (37 patients and 87 caregivers). The median participation was 5.6 individuals per class. Of those whom have participated in the online class and completed the post education class survey 83.4% have rated their overall satisfaction 4‐5 out of 5. Furthermore, online surveys, they expressed much pleasure with mode of delivery, device used, and other technical aspects of using the online software application.

Discussion: Telehealth has been identified as an integral aspect of the strategic plan and delivery of transplant services. The performance improvement project confirms the added value of online patient and caregiver education in meeting the gap in care for distant learners. These activities support the transplant program’s alignment with regulatory requirements, and informed consent, and the imperative of delivering patient and family centered care. The pilot project continues its work by bringing education to distant learners and has served as an example for other interdisciplinary telehealth education and support endeavors. Further study will focus on expanding the current education delivery model to include multidisciplinary material, content, and provider appointments.

34. Tele‐behavioral Health Services in Rural School Settings Before and During COVID‐19

Marcia M. Ward PhD,¹ Fred Ullrich BS,¹ Kimberly A. S. Merchant MA,¹ Knute D. Carter PhD,¹ Divya Bhagianadh MPH,¹ Meghan Lacks PhD,² Erika Taylor MS,² Jennifer Gordon BS³

¹University of Iowa ²East Carolina University ³Bay Rivers Telehealth Alliance

Background: Over 22% of children and adolescents in the U.S. have had or currently struggle with a mental or behavioral health disorder with severe impairment. Receiving treatment for these conditions remains problematic due to a profound shortage of behavioral health providers in the U.S. The utilization of tele‐behavioral health services is an effective method to provide quality care. School‐based tele‐ behavioral health services offer considerable success at reaching youth, but the research literature on the use of these services in schools is limited. Data from 15 school‐based telehealth programs across the U.S. are examined to demonstrate the role tele‐behavioral health can play in providing access to behavioral health services in rural youth. Furthermore, data on services before and during the COVID‐ 19 public health emergency (PHE) are examined to elucidate how school‐based tele‐behavioral health services responded to the PHE.

Methods: In September 2016, the HRSA Office for the Advancement of Telehealth (OAT) awarded grants to 21 organizations across the country for the School‐Based Telehealth Network Grant Program (SB TNGP). This program was designed to demonstrate how telehealth can expand access to and improve the quality of healthcare services offered in schools. As part of this initiative, OAT funded the Rural Telehealth Research Center (RTRC) to serve as a data coordinating center for the program. RTRC collected data on all students receiving tele‐behavioral health services in each of four semesters. All SB TNGP schools paused in‐school instruction at some point in Spring 2020 due to the PHE and many grantees adjusted services at that point. To understand these changes, we included schools that provided services in the Fall 2019 semester and the Spring 2020 semester and termed these Fall 2019 continuing schools. Then, we split the Spring 2020 data into two cohorts – continuing schools and new schools. The three resulting groups were compared using Chi‐squared tests. Statistical significance was set at p<05.

Results: From Fall 2019 to Spring 2020 the 62 continuing schools increased the number of students served from 396 to 745 and the average number of encounters per student from 2.4 to 4.1. The 25 new schools averaged 6.5 encounters for the 301 students receiving services. For continuing schools, the percentage of encounters delivered by psychiatrists or professional counselors decreased significantly (p<.001), while the percentage of encounters delivered by clinical social workers, mental health counselors, and clinical psychologists increased significantly (all p<.001). For the new schools, most encounters were delivered by clinical social workers and professional counselors. For continuing schools, the percentage of encounters involving individual counseling, family counseling, or group counseling increased (p<.001) and decreased for medication management (p<.001). For new schools, most encounters involved individual counseling and/or assessment and/or coordination of services. Due to the PSE, for continuing schools, the percentage of students released back to school decreased while the percentage released to parents increased significantly (p<.05). For new schools in Spring 2020, almost all students were released to parents.

Discussion: Data indicate school‐based tele‐behavioral health is an effective care delivery model prior to and during the PHE in rural settings. Both continuing (62) and new (25) schools associated with the SB TNGP grantee cohort successfully utilized telehealth to initiate and follow‐up on established and newly emerged behavioral health needs remotely, as indicated by marked increases in encounter rates during the Spring 2020 semester. Data indicate grantees increased clinical staff or re‐assigned pre‐existing staff (e.g., clinical social workers, clinical mental health counselors, and clinical psychologists) to account for increased demand and changing clinical landscapes. Additionally, these changes prompted an increase in individual, family, and group tele‐behavioral health encounters. Overall, data indicate grantees responded swiftly to the PHE by greatly expanding service coverage in continuing and new schools and number of students served through the delivery of tele‐behavioral health.

35. Telehealth‐Delivered Intensive Behavioral Autism Interventions during COVID‐19: Two Case Studies

Alice Zhang PhD, Makenzie Danley, Linda Heitzman‐Powell

University of Kansas Medical Center

Background: Early Intensive Behavioral Intervention (EIBI) based on applied behavior analysis (ABA) as an effective treatment for children with Autism Spectrum Disorder (ASD) has traditionally been delivered through an in‐person model. Telehealth delivery of ABA services and its research focused exclusively on training and coaching staff and caregivers in using ABA skills. In response to the constraints of the COVID‐19 pandemic, ABA clinicians quickly adapted and transitioned to provide EIBI services via Telehealth.

Methods: An archival data analysis or record review was conducted to examine the skill acquisition of two Hispanic children who receive EIBI services during center‐based settings and Telehealth settings at a regional medical center.

Results: Before COVID, two Hispanic children with ASD received EIBI services including 20‐25 hours of one‐on‐ one interaction with a behavioral technician; 2 hours of parent training, and 5 hours of protocol modification and supervision of technicians each week. Child A received 72 days of center‐based EIBI services between November 2019 and March 2020 and 35 days of Telehealth‐based services. Child B received 31 days of center‐based EIBI services between January and March 2020 and 47 days of Telehealth‐based services. The dosage of one‐on‐one teaching significantly decreased, protocol modification increased, and parent training decreased during the Telehealth phase compared to the center‐based service phase for both children. Skill acquisition before COVID‐19, went from an average rate of 2.0 ± 3.2 to 1.6 ± 1.7 per day after COVID‐19 in Child A and from an average rate of 1.4 ± 2.2 to1.5 ± 2.6 per day in Child B. Child A learned 202 skill targets over 107 days while Child B learned a total of 113 skill targets over 78 days.

Discussion: The use and research of Telehealth‐delivered EIBI services can be beneficial to not only address the pandemic concerns but also to bridge the service gap experienced by those in rural and underserved communities. Lessons learned from these two cases regarding telehealth setup, cultural consideration are discussed.

36. QI in Pediatric Complex Care: Growing and Sustaining a Telehealth Practice during COVID‐19

Ellen McAndrews Guth MSN, Tina Smith CTC CTL

UF Health Jacksonville/ University of Florida Jacksonville Physicians Inc., UF College of Medicine

Background: Our objective was to use the necessary changes in healthcare delivery during the COVID‐19 pandemic to accelerate ongoing quality improvement initiatives in growing telemedicine services in our primary care practice that serves children with medical complexities. We believe synchronous and asynchronous telehealth care greatly benefits the patients and families of our highly complex pediatric population. However, despite our efforts, participation of families and staff members was low. Our goal was to use this COVID‐19 catalyst to create an enduring and sustainable telehealth presence and practice, and to do so, we invested in meeting both patient and staff needs.

Methods: We used the Model for Improvement to guide PDSA cycles with the aim of increasing and sustaining telehealth usage by patients and staff. This would be achieved by increasing asynchronous patient portal contacts and finding equilibrium in virtual visits and office visits (compared to pre‐COVID utilization). Success of incremental improvements was measured in feedback from staff and patients (both anecdotally and with surveys) and in tracking IT service requests for virtual visit support as compared to the number of virtual visits completed.

Results: Patients reported increased comfort in using the patient portal and confidence in participating in virtual visits. The percentage of patients completing office visits active to our patient portal increased from 72% pre‐COVID to essentially 100% during (and ongoing). Patients reported high satisfaction with virtual care and planned to use this option whenever possible and appropriate. Staff demonstrated high levels of comfort with using the patient portal, confidence in conducting virtual visits, and working in remote environments. Completed patient appointments have maintained an even split between in‐clinic and virtual visits since August 2020. IT service requests compared to virtual visit appointments peaked in April 2020 and maintained high levels over the course of the spring. They have returned to pre‐COVID levels while virtual visit appointments are at a 10% increase from their pre‐COVID averages across our larger organization.

Discussion: We were able to maintain a level of patient and staff satisfaction, develop additional care coordination methods, and increase and sustain the percentage of virtual visits conducted. We had no clinic closures related to COVID‐19 and little to no interruption of patient care. The user experience is important. To successfully implement broad practice change, we must remember both patients and staff are the target audience. If both groups feel supported and championed, it elevates everyone’s experience.

37. Virtual Care and Telehealth at Health Centers Reported in the Uniform Data System

William L. England PhD JD

Health Resources and Services Administration

Background: For 55 years, health centers have delivered quality primary care to millions of people, regardless of ability to pay. Currently, HRSA funds nearly 1,400 health centers and almost 13,000 service delivery sites for nearly 30 million people in every state and territory. Prior to the Public Health Emergency (PHE), 43% of health centers reporting using telehealth (plus 8% considering telehealth), but usage volume was quite low at 478,333 (0.4%) of 122,303,749 visits in 2019 (range 4% to 0.02% by state). The primary impediment to using telehealth was Medicare or other payers that did not allow health centers as distant site providers. That impediment disappeared with the PHE, as limitations for telehealth were waived. In August 2021, HRSA will release Uniform Data System (UDS) results for health center service in 2020, providing the first snapshot of telehealth in health centers in response to the PHE. This presentation will analyze UDS telehealth data for 2020, compared to prior years.

Methods: The UDS survey first began asking about telehealth use by health centers in 2016. Additional questions have been added each year and in 2019, health centers were first asked to report their volume of virtual visits. However, because visit volumes by service type might reveal proprietary information relevant to business or financial applications, health centers may request suppression of visit volume in public UDS disclosures. In 2019, about 63% of health centers requested non‐disclosure of visit volumes and aggregated volume data was only available at the state level. While that may limit detailed analysis of virtual care at individual centers, aggregate data will capture the dramatic change in telehealth use at health centers in 2020.

In addition to analysis of quantitative data on telehealth use by health centers, UDS also asks health centers to report challenges with telehealth such as reimbursement, broadband, cost of equipment, patient and provider acceptance, and other reasons they may not use telehealth. Comparison of 2020 answers to prior years will reveal perceptions and changes regarding telehealth under the PHE.

Results: Because UDS 2020 data will not be released until mid‐August, results are not yet available. Voluntary weekly surveys of health center telehealth use during the pandemic showed that all responding centers were using telehealth by May 2020, comprising over half of their total visits. There was substantial variation in percent telehealth use in May, ranging from 88% in Connecticut to 18% in South Dakota. By October, telehealth visit volume had dropped to 27% of total visits, with 5% of centers reporting they were no longer using telehealth.

Discussion: Because health centers receive payment from a wide range of sources, pre‐pandemic Medicare limits on serving as a distant site telehealth providers did not necessarily apply to Medicaid or other payment, so reimbursement was not the only reason for low use of telehealth prior to the pandemic. This analysis will examine state policies regarding telehealth and correlate that with telehealth use at health centers. Prior publications such as Key Adoption Factors, Barriers, and Opportunities for Telehealth in Health Centers showed that rural location, operational factors, patient demographics, reimbursement policies and technical issues influenced health centers’ decisions about using telehealth. This work will examine such issues in light of pandemic motivated policy changes and plans by states to extend or make permanent those changes.

38. Telehealth Research and the Health Resources and Services Administration

William L. England PhD JD

Health Resources and Services Administration

Background: HRSA serves as the primary federal agency for improving health care to people who are geographically isolated, economically or medically vulnerable. HRSA programs help those in need of primary care or people with special health care needs. HRSA also supports improvements in health care delivery and training of health professionals, especially providers for underserved areas. HRSA does this largely through grant funding, which is 90% of HRSA’s $12B annual budget. In recent years, telehealth or virtual care has been increasingly featured in HRSA funding opportunities. In 2020, nearly all HRSA awards allowed for the use of telehealth in service delivery. While HRSA is not a research agency, HRSA’s Office for the Advancement of Telehealth (OAT) has increasingly been seeking to fund telehealth research and to increase the evidence base for telehealth as part of its mission to advance telehealth. This presentation will focus on OAT and HRSA support for telehealth research.

Methods: Starting with HRSA’s first telehealth project inventory in 2017, HRSA has annually asked project officers to report awards that mention telehealth (or words related to telehealth) in funding opportunity notices or applications. Not surprisingly, mention of telehealth was growing exponentially even before the COVID‐19 Public Health Emergency (PHE). The CARES Act and other legislation substantially increased flexibility and funding for the use of telehealth and HRSA was able to add new projects both to expand telehealth service and to study telehealth service delivery. This presentation will summarize those new programs and work.

Results: Since 2017, HRSA’s inventory of telehealth grantees has grown from 1,000 grantees to almost 4,000 grantees having an award that mentioned telehealth or related activity. This presentation will discuss the growth of projects by Bureau and type of service or clinical telehealth use. Also in 2020, OAT doubled its telehealth focused research capacity from one center to two (the Rural Telehealth Research Center and the Rural Telehealth Evaluation Center) and created a new telehealth website (Telehealth.hhs.gov) for the Department of Health and Human Services. The new federal telehealth site initially had a tab “For Patients” and a tab “For Providers” with information on the use of telehealth for the PHE. Recently a third tab “For Researchers” was added. This presentation will highlight that research tab and discuss OAT plans for expansion. This presentation will also highlight other HRSA funded resources to support telehealth research including the Rural Health Information Hub (RHIhub), the Rural Health Research Gateway, the Telehealth Centers of Excellence and Telehealth Resource Centers (TRCs), with research enabling resources such as the Northeast Telehealth Resource Library or the Policy TRC 50 State Telehealth Survey.

Discussion: HRSA’s primary contribution to telehealth research is support of the telehealth research community. This presentation will highlight the growth and breadth of HRSA support for telehealth projects and research and may elicit comments to improve those efforts.

40. Strategies for Adapting an Evidence‐Based Breast Cancer Intervention During the COVID‐19 Pandemic

Michelle S. Williams PhD,¹ Sheila Y. McKinney PhD,² Precious Ugwu¹

¹George Mason University, ²Jackson State University

Background: Evidence‐ and theory‐based breast cancer education programs have been proven to be effective tools for increasing breast cancer screening use. The Witness Project is a highly effective breast cancer education program that is targeted toward African American women. One of the core components of the face‐to‐face program is a testimonial by a breast cancer survivor who testifies about her breast cancer journey. The Intervention Mapping Adapt framework was used to guide the adaptation of the Witness Project into an online program. For our adapted intervention, the Deep South Witness Project, we developed a series of 9 brief videos with messages from local African American breast cancer survivors to replace the live testimonials.

Methods: A multiphase mixed methods design was used to conduct the study. During Phase II of the study, we conducted a needs assessment with African American breast cancer survivors (n = 9) to identify barriers to breast cancer screening that were unique to African American women in Mississippi and Alabama. A phenomenological approach was used to develop the interview guide. During the interview participants were asked to discuss their experiences being diagnosed with and treated for breast cancer and to identify barriers that they believe prevent African American women from getting breast cancer screenings regularly. Participants were also asked to share messages intended to motivate African American women to get breast cancer screenings. Recruitment was stopped once data saturation was achieved. The qualitative data was analyzed by two members of the research team who conducted two cycles of coding to identify common themes.

Results: The mean age of our participants was 59.4 years old. The most common themes that emerged from the interviews were psychosocial barriers to breast cancer screening (ex. fear of painful mammograms, fear of finding cancer), beliefs about breast cancer (ex. myths about the causes of breast cancer), and the general lack of knowledge about breast cancer among African American women. During the interviews, we encountered some unexpected challenges with the participants’ technology or their ability to use technology. As a result, the video series was created by combining audio clips from the interviews with stock photos of African American women. The 9 videos ranged in length from 34 seconds to 1 minute and 58 seconds. The topics of the videos included: the importance of early detection, the perceived risk of breast cancer, the need to be aware of breast changes, and the importance of mammograms. The videos were embedded in the Deep South Witness Project presentation that was delivered online to 37 pilot study participants in Phase III of the study. The qualitative feedback from the pilot study participants indicated that the videos were informative, enlightening, and impactful.

Discussion: The results of our study indicate that it is feasible to adapt an evidence‐based intervention to be delivered online. Core components of the intervention may be modified by without compromising the effectiveness of the intervention. Delivering cancer education interventions online may be an important strategy for increasing the reach of the programs to underserved populations. However, limitations imposed by the digital inequities that exists in underserved communities can be a challenge. More resources are needed to ensure that underserved populations have the informational and instrumental resources that are needed to be fully engaged in online environments.

41. An Evaluation of the Financial Performance of Rural Hospitals Providing Telehealth in Emergency Departments

Dunc Williams Jr. PhD, Mary Dooley PhD, Jada M. Johnson MS

Medical University of South Carolina

Background: The University of Mississippi Medical Center (UMMC) has operated a robust telehealth emergency department (ED) network, TelEmergency, since 2003. TelEmergency enhances access to emergency medicine‐trained physicians at participating rural and critical access hospitals. TelEmergency was developed as a cost‐control measure for financially‐constrained rural hospitals to improve access to quality, emergency care. However, the literature remains unclear as to whether ED telehealth services are associated with better hospital financial performance. Our objective is to empirically determine whether TelEmergency was associated with changes in myriad financial performance indicators at participating hospitals when compared to similar hospitals without TelEmergency between 2010 and 2019.

Methods: A panel of data for 2010 – 2019 was constructed at the hospital‐level. Hospitals with TelEmergency (n = 19 hospitals; 185 hospital‐years) were compared to similar Mississippi rural or critical access hospitals that did not use TelEmergency (n = 44; 311 hospital‐years). To date, unadjusted statistics describing the relationship between myriad financial performance indicators and treatment (i.e., participation in TelEmergency) have been evaluated, as well as the relationship between pre‐ vs post‐ TelEmergency participation. Going forward, the relationship between myriad financial performance indicators and treatment will be predicted using generalized estimating equations (GEE) with robust standard errors.

Results: Unadjusted and for the study period, TelEmergency hospitals performed better than similar Mississippi hospitals that did not participate in TelEmergency across some financial indicators. TelEmergency hospitals reported better profits (i.e., total and operating margins), slightly better (lower) days in accounts receivable, but worse (lower) days cash on hand.

Unadjusted and compared to pre‐participation in TelEmergency, participant hospitals reported similar total and operating profits, notably better (lower) days in accounts receivable, but materially worse (lower) days cash on hand.

Discussion: Unadjusted statistics show that TelEmergency hospitals reported better profits than similar Mississippi rural or critical access hospitals that did not use TelEmergency; though, TelEmergency hospitals did not improve profits after implementing TelEmergency. These statistical comparisons could mean many things. It will be best to interpret them in the context of our forthcoming regression‐adjusted analyses.

Overall, the potential for TelEmergency to impact the financial performance of rural and critical access hospitals is incredibly important to the future availability of acute care in rural America. In an environment of increasing rural hospital closures and with over a third of all rural hospitals deemed financially at risk of closure, tele‐emergency programs such as TelEmergency may be one opportunity to contribute to the stabilization of some rural hospital finances and maintain access to acute care in those areas.

42. Synthesizing Telehealth Research to inform Healthcare Decisions: the AHRQ Evidence‐based Practice Center Program

Christine Chang MD, Lionel Banez MD, Craig Umscheid MD, David Niebuhr MD, Suchitra Iyer PhD

Agency for Healthcare Research and Quality

Background: The Agency for Healthcare Research and Quality (AHRQ) Evidence‐based Practice Center (EPC) Program has commissioned systematic reviews, scoping reviews, evidence maps, and other evidence reports since 1997, and published one of its first reports about telehealth in 2006. There has been increased interest in telehealth, telemedicine, mhealth, and other related technologies since the COVID‐19 pandemic, with an emphasis on evidence about implementation and optimization of telehealth to inform decision‐making. In response to COVID, the EPC Program has commissioned new reviews to address urgent questions including those about telehealth and other related technologies and its impact on patient health outcomes.

Methods: The EPC Program commissions reviews in response to the needs of the public and Federal agencies. It partners with external organizations and Federal agencies to promote evidence‐informed healthcare decision‐making. Through engagement of end‐users and stakeholders and established evidence synthesis methods the EPC Program is developing evidence reports with diverse synthesis methods that are rigorous, relevant, and timely to inform healthcare decision‐making and future research by a variety of end‐users. Since March 2020 the EPC Program has published three evidence reports and started six additional reports related to telehealth. These cover a range of topics including: telehealth implementation, patient and provider experience, patient‐generated health data (PGHD), interventions to promote healthcare engagement, barriers to and facilitators of use by rural populations, use in specific clinical scenarios such as antenatal care, behavioral health for cancer care, and pain management programs, and mobile app selection frameworks.

Results: Findings across EPC reports include: positive effect of PGHD on outcomes for coronary artery disease, heart failure, and asthma; provider‐to‐provider telehealth in rural settings can improve access and quality of care, knowledge and self‐efficacy of providers, and reduced costs to rural providers; mobile app selection frameworks do not account for potential harms of apps and potential risks posed by machine learning and artificial intelligence; and telehealth benefit when used to expand critical care and speed emergency care decisions, and replace much face‐to‐face care. Common recommendations include: recruiting diverse populations and reporting social determinants of health; need for longer rigorous studies of health outcomes; attention to implementation issues including patient adherence, provider workflow, and health literacy; need for larger, more rigorous observational studies, quasi‐experimental studies and more trials (including adaptive trials) in rural settings; improved examination of the impact of telehealth and mHealth when used in the context of multicomponent interventions; and the need for new frameworks for evidence generation that can keep up with rapidly evolving technologies.

Discussion: Evidence prior to COVID can reassure end‐users that telehealth can benefit patients and now has an added benefit of reducing exposure to infection. The explosion in telehealth research and experience under COVID can inform ongoing and future evidence reviews. In‐process telehealth evidence reviews such as telehealth under COVID19, women’s experience with telehealth, and telehealth for antenatal care can further inform next steps in implementation and research. Evidence synthesis on questions beyond effectiveness, focusing on implementation, barriers and facilitators, patient experience, and other questions can optimize the use of telehealth and other technologies for patient care. Systematic reviews and other evidence synthesis can identify research gaps to inform future telehealth research to close these gaps.

43. Towards Machine Learning Assisted Strep Throat Diagnosis

Samay Shah,¹ Ahmed Allabban MBBS,² Manuel Rebol,³ Neal Sikka MD, Colton Hood MD¹

¹George Washington University School of Medicine and Health Sciences, ²King Abdulaziz University at Jeddah, ³American University

Background: Streptococcus pharyngitis, or strep throat, is a very common reason for ambulatory care visits in the United States annually. Early and accurate detection of strep throat helps improve outcomes and minimize complications. Currently, the Centor criteria, a set of validated clinical decision rules, are widely used by clinicians to guide management. It is anticipated that medical care will more frequently be delivered by telehealth during and following the COVID‐19 pandemic, requiring novel approaches to disease diagnosis. To enhance the applicability and potential diagnostic capability of the Centor score, our goal is to study a machine learning algorithm that pairs the elements of the Centor score with images, videos, and audio recordings of throats to determine whether the patient has streptococcal pharyngitis.

Methods: Participants were recruited via a convenience sample. All participants presented to the emergency department or urgent care with a chief complaint of sore throat. Three elements of audiovisual data were recorded: an image and a brief video of the participant’s throat, and a recording of the participant saying “I have a sore throat.” Our pharyngitis detection algorithm aims to augment the Centor score. We perform supervised learning on the parameters of the convolutional neural network architecture ESPNet. Our neural network is trained on two tasks: semantic segmentation of the throat and classification between healthy throat and pharyngitis. The encoder‐decoder model learns to segment the region of interest for the diagnosis: tonsils, uvula, and other throat landmarks. For classification, we reuse the encoder parameters trained for semantic segmentation and fine‐tune the parameters for the binary classification of pharyngitis. In addition to the visual analysis, we extract the mel‐frequency cepstrum coefficients from the audio and pass it to a ResNet‐50 model pre‐trained on voice data to classify pharyngitis.

Results: We conducted a preliminary review of 37 participants enrolled in the study with complete data collection. Participants had an average age of 31.4 years. In our sample 3 patients had a Centor score of ‐1, 10 had a score of 0, 10 scored 1, 9 scored 2, 4 scored 3, and 1 patient had a score of 4. 32 patients received strep tests, of which 5 were positive and 27 were negative.

We assessed the quality of our visual data by classifying it via Mallampati‐like score in which the patient phonated during data collection. Though the Mallampati score is typically used for assessment of airways, in this case it provides a quantifiable way to assess the quality of images that will be used in our model. Scores range from 1 to 4, with 1 indicating a clear and complete view of the soft palate, and 4 designating that the soft palate is not visible at all. Currently 15 patients have a score of 1, 12 have a score of 2, 7 scored 3, and 3 scored 4. The device’s flash was active for all 37 patients. Patient recruitment continues, and we plan to present our results based on a larger data set.

Discussion: As more care is delivered via telehealth, clinicians will require diagnostic aids to help triage those that require subsequent in‐person evaluation, which may reduce cost effectiveness, efficiency and convenience of telehealth. This study aims to develop a machine learning algorithm that can help clinicians better distinguish between healthy throats and pharyngitis by using audio and visual data augmenting the Centor criteria. Machine learning has previously been used to analyze images of the tonsil to identify pharyngitis. Our study combines clinical parameters of the Centor criteria with machine learning analysis of visual and audio data to improve clinical decision support.

44. Evaluating usability and feasibility of a remote exam device for pediatric complex care patients

Marie Pfarr MD, James Odum MD, Kathleen Pulda, Jennifer Ruschman, Kenneth Tegtmeyer

Cincinnati Children’s Hospital Medical Center

Background: Children with medical complexity (CMC), defined as children with chronic, functional limitations and technology dependence, account for high health care utilization. Telemedicine holds significant potential in CMC, as it allows a provider to engage with CMC in their home environment and can alleviate both financial and transportation burdens. Remote exam devices that enable the performance of a high‐fidelity physical exam could expand the ability of providers to clinically assess their CMC patients during a telemedicine visit. In this pilot study, our objective was to determine the usability and feasibility of a novel remote device, which consists of an infrared thermometer, a digital stethoscope, a digital otoscope, a tongue depressor, and a built‐in camera.

Methods: The remote exam device was distributed to caregivers of CMC cared for at a tertiary institution’s complex care outpatient center. Caregivers were excluded if their residence lacked Wi‐Fi or if the caregiver was not English proficient due to the inability to provide interpreter telemedicine services at this time. The remote exam device was integrated into an encounter using an institution specific telemedicine platform. Surveys were administered via REDCap to both caregivers and providers after each telemedicine encounter where use of remote device was attempted. Usability data was analyzed using the System Usability Scale (SUS) adapted specifically to the device. Feasibility was studied at the individual encounter level focused on (1) Successful connection between CMC/caregivers and providers using designated telemedicine platform (2) Successful integration of device and the telemedicine platform and (3) Successful connection to any of the device’s adapters that the provider wished to use during the encounter.

Results: Thirty‐eight caregivers participated in the pilot study with a total of 65 encounters completed. Post‐encounter caregiver survey response was 52.7% and post‐encounter provider survey response was 93.5%. Using the SUS, the average caregiver reported usability score was 86.7 which categorizes the device as “excellent usability”. In assessing the feasibility of the device in this cohort, approximately 6% (4/65) of visits had problems connecting CMC/caregiver and provider to the telemedicine platform, resulting in 2 cancelled visits. Out of the 63 encounters that were successfully able to connect to the telemedicine platform, 28.5% (18/63) of these encounters had problems integrating the remote exam device and platform, resulting in an additional eight visits being cancelled. Out of the remaining 55 visits, 9.1% (5/55) had difficulty connecting to desired device adapters, leading to two encounters ending early. Overall, 81.5% (53/65) encounters were successfully completed.

Discussion: The remote exam device was rated as having excellent usability by caregivers; feasibility was notably more problematic with technology issues contributing to several encounters being cancelled or ending early. Future interventions will focus on improving connectivity rates and accessibility to a non‐English speaking caregivers. Data from this pilot study supports the use of this novel remote exam device in CMC with the ultimate goal of assessing the impact of this device on outcome and utilization metrics.

45. An mHealth application to manage pediatric asthma: Use and benefits

Kandia Lewis PhD, Cynthia Zettler‐Greeley PhD

Nemours Children’s Health

Background: Pediatric asthma is a serious chronic disease of the lungs affecting millions of children in the US. Mobile health (mHealth) technologies may afford patient‐families and providers a means to successfully manage pediatric asthma. This study examined an mHealth application designed to educate, engage, and facilitate access to care. The Nemours app features include telehealth video visits, medication reminders, images of asthma medications and videos to demonstrate proper medication use, asthma education resources, digital tracker to record asthma symptoms, interactive asthma action plan, air quality index, and messaging system. The purpose of this study was to examine: 1) which app features clinical providers and patient‐families used and how frequently they used them and 2) whether app use yielded benefits for clinical providers and patient‐families who used it for asthma management.

Methods: Nine allergists and/or pulmonologists and 80 patient‐families who had a child with asthma aged 5‐11 years enrolled in the study. Active study participation for patient‐families and providers lasted 6 months, encompassing two time points (Time 1— baseline, and Time 2— study conclusion), during which data were recorded on access and feature use of the app by patient‐family study participants. Provider participation involved communicating with patient‐families and accessing patient app data via the app’s clinical dashboard. At Time 2, providers completed a brief digital survey about their app use. A retrospective patient asthma health history (e.g., asthma control test (ACT) scores, telehealth visits, and urgent care visits) was obtained two years prior to study participation. Retrospective app use two years prior to study participation also was extracted to inform baseline levels of app logins and feature use. At Time 1, patient‐family participation involved completion of digital surveys about demographic information and app use and completion of a health literacy phone screener. Six months later, at Time 2, patient‐families completed another digital survey about app use and participated in another health literacy phone screener.

Results: Five of seven providers who completed the survey reported they used the app during the 6‐month study. Providers reported forgetting and not having enough time as barriers to accessing patient app data. Eighty percent of providers who used the app reported using it occasionally and 60% reported using it prior to seeing patients. Additionally, 60% of providers who used the app reported they reviewed patient‐family asthma tracker data, 40% reviewed messages, and 40% reported using app patient data to inform their clinical decisions. According to app tracking records, 61% of patient‐ families used the app (n = 49). Patient‐families who used the app reported using the app occasionally (36%), monthly (27%), and weekly (36%), with 77% reporting use of the app in the evening. The asthma tracker (66%), messages (64%), and telehealth (41%) were the features most reportedly used by patient‐families. Caregivers reported use of the app message feature was a significant and positive predictor of caregiver health literacy gains (b = .441, p = .041); however, total tracker entries completed was not (estimate = ‐.001, p = .074). Finally, those who used the app tended to have children whose asthma was poorly controlled (r = ‐.39, p<.01).

Discussion: The findings revealed benefits of app use for providers and patient‐families who used it. Two providers used the app to inform their clinical care decisions regarding their patients, demonstrating high value in provider app utilization despite low provider engagement in app use overall. A majority of patient‐families were identified as app users, and most reported using the app at least monthly. Patient‐families were more likely to use the app when their child’s asthma was poorly controlled, highlighting a key indicator in its engagement among patient‐families. Results demonstrated that caregivers’ use of the messaging system predicted health literacy gains evidenced at Time 2.

Improved health literacy is beneficial because caregivers with strong health literacy are more likely to have children whose asthma is well controlled (Harrington et al., 2015). Future research should examine ways to overcome utilization barriers for providers and patients to best maximize mHealth app benefits.

46. Telemedicine in the Continuum of Neonatal‐Perinatal Care: Current State and Opportunities

Abeer Azzuqa MD,¹ Jeanne Zenge MD,² Rashmin C. Savani MBChB,³ Kerri Machut MD,⁴ Sonja Ziniel PhD,⁵ Jawahar Jagarapu MD,³ Abhishek Makkar,⁶ John Chuo MD⁷

¹University of Pittsburgh School of Medicine/ UPMC Children’s Hospital of Pittsburgh, ²University of Colorado School of Medicine and Children’s Hospital of Colorado, ³The University of Texas Southwestern Medical Center/ Children’s Medical Center, ⁴Northwestern University Feinberg School of Medicine and Ann & Robert H. Lurie Children’s Hospital of Chicago ⁵University of Colorado, ⁶The University of Oklahoma Health Sciences Center, ⁷Children’s Hospital of Philadelphia

Background: Telemedicine is a valuable tool with potential opportunities to deliver and/or enhance comprehensive family‐centered and specialized multi‐disciplinary neonatal care when appropriate. A knowledge gap exists regarding the current state of telemedicine use and study methods in neonatal‐perinatal telehealth, as well as opportunities for future collaboration across the Children’s Hospitals Neonatal Consortium (CHNC) sites. This cross‐sectional study intended to identify the various telemedicine applications in clinical and educational settings, and to assess individual program characteristics such as maturity, staffing, and perceived barriers for implementation. It also assessed whether the areas being actively studied constituted quality improvement or research.

Methods: A telemedicine survey was designed by members of the CHNC Neonatal Perinatal Digital Health Special Interest Group and reviewed by a Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) funded survey methodologist. Domains included characteristics of the NICUs, staffing, barriers, clinical use, education, technical/system topics being actively studied as quality improvement or research. The clinical use questions measured the use and maturity of virtual activities at different stages of a neonate’s typical longitudinal clinical journey – prenatal care, stabilization at birth, transfer to a higher level of care, NICU hospitalization, and transition to home.

The research domain asked respondents about study design methods they use or are interested in using and what parts of the infant’s clinical care journey they would be interested in studying telemedicine as a CHNC collaborative project. The web survey was distributed electronically to the site leaders at each CHNC center (n = 35) between October 2020 and January 2021. Data was collected in REDCAP software and was further analyzed using descriptive statistics.

Results: Overall, 28 out of 35 centers responded to the survey, a response rate of 80%. Data indicated that a growing number of telehealth initiatives at various levels of maturity had been developed across the CHNC centers, particularly in prenatal consultation, virtual rounds, subspecialty support, tele‐lactation, and transition to home (Figure 1). Additionally, tele‐education activities focused on provider didactics, case reviews and simulation had either existed or were implemented during the COVID pandemic (Figure 2). For the study domain, the highest study activities were in the areas of individual experience (38%), the quality of healthcare delivery (31%), and lessons from implementation (28%).

Only 13% of respondents reported actively studying specific patient or population health outcomes. The most used methods were feasibility and quality improvement designs, while most “interested but not used before” methods centered on randomized controlled studies, cohort, and interrupted time series (Figure 3). Potential study areas of most interest were transport communication, prenatal consultation, delivery plans discussions with families facing complex conditions, family rounds, and family/clinician updates.

Discussion: This is the first study to explore the current state of neonatal telehealth among CHNC centers. The results show the broad application of telehealth initiatives in clinical and educational settings across the continuum of neonatal care from fetus to post‐discharge. In addition, this study provides baseline data for CHNC sites which can be used to prioritize study areas and methodologies and to start multicenter efforts studying telemedicine as a novel care modality that can impact neonatal health delivery quality, cost‐effectiveness, and patient outcomes. It also suggests a potential to address health care disparities and a move towards a cost‐effective, efficient, and value‐based care model.

Lastly, the study illuminates the desire not only for quality improvement initiatives but also research efforts using multicenter randomized trials, cohort and interrupted time series designs to understand the effectiveness of these programs and their impact on health care outcomes.

47. A framework for designing, and deploying telehealth equity dashboards in health care organizations

Marlíse Arellano AB,¹ Erica Wu,² Jessica Zhang MPH,¹ Sansanee Craig MD,³ Suzinne Pak‐Gorstein MD,⁴ Phil Scribano DO,³ Stormee Williams MD⁵

¹Boston Children’s Hospital Innovation & Digital Health Accelerator, ²Harvard University, ³Children’s Hospital of Philadelphia, ⁴Seattle Children’s, Children’s Hospital of Philadelphia, ⁵Children’s Health

Background: The COVID‐19 pandemic catalyzed a surge in telehealth utilization and adoption. However, it also brought to light systemic disparities experienced by underserved populations in accessing digital technologies in telehealth. To better understand the extent of disparities experienced by underserved populations, health care organizations have begun to create and deploy analytics dashboards to more robustly measure, track, and intervene upon telehealth metrics related to access and utilization, specifically for underserved populations. However, there is limited literature and guidelines on creating such dashboards. In this presentation, the authors propose a plan for designing and implementing telehealth equity dashboards in health care organizations, collating key findings from identified major sources, including a cross‐institution survey and follow‐up interviews.

Methods: The Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) Telehealth Equity Dashboard Working Group conducted an online, voluntary survey sent out to all institutions part of SPROUT. The Working Group includes pediatric academic medical centers from every geographic region and includes both urban and rural facilities. The purpose of this survey was to better understand current practices and content included in telehealth equity dashboards, common barriers faced in dashboard development and addressing telehealth disparities, and similarities in demographics across respondents (e.g., types of patients served). After conducting the initial survey, a cohort of the Working Group conducted follow‐up qualitative interviews to more deeply understand dashboard development. The framework and recommendations in this presentation were identified by leveraging survey and interview responses as well as additional literature reviews and discussion in the Working Group.

Results: The authors propose a standard framework for the development of a telehealth equity dashboard in four key phases: Assess, Define, Design, and Implement. This framework and process is not necessarily linear, but encapsulates key steps and considerations for institutions to consider when developing a telehealth equity dashboard. In our results, the authors include recommendations, guidelines, and relevant findings on:

• Making the case for equity to stakeholders

• Mapping out institutional data sources, identifying gaps, and brainstorming solutions to address data quality issues

• Defining use cases and scenarios for further tailoring the telehealth equity dashboard, including guidelines on equity stratifier considerations

• Identifying specifications for visualization practices in a dashboard

• Effectively deploying a dashboard in an institution, including socialization

• Determining short and long term interventions based on telehealth equity dashboard results

Discussion: In this presentation, the authors propose a strategy for creating dynamic and comprehensive dashboards to track telehealth equity. Proposed guidance includes, but is not limited to, how to tailor dashboards to audience types and patient populations and how to develop dashboards in light of limited data. This work is essential to support future development and adoption of telehealth equity dashboard standards across healthcare organizations. Beyond dashboard creation, there is a need for collective action to further prioritize more thorough and comprehensive collection of patient demographic data including Race, Ethnicity, and Language data (REaL). The authors propose additional collaboration for institutions to more readily share learnings on addressing telehealth equity as well as real‐time and longitudinal data and metrics to better understand and address disparities collectively.

48. Community‐Engaged Research Advisory Board Adaptations During the COVID‐19 Pandemic

Sarah Rhoads PhD,¹ Patty J. Reed Med,² Heather Marshall BSN,² Steve Bruno,³ Leanne Lefler PhD,² Tiffany Haynes PhD²

¹University of Tennessee Health Science Center, ²University of Arkansas for Medical Sciences, ³Community Research Partner

Background: Community engagement is recognized as an important part of clinical and translational research. However, with the stay at home restrictions in March 2020 and continued social distancing guidelines, engaging with community stakeholders in a meaningful way became a new obstacle. Many research projects moved engagement activities, such as advisory board meetings to video conferencing platforms such as Zoom or Microsoft Teams. While research faculty and staff may have been familiar with video conferencing, many lay community members were less comfortable with this technology; thus, potentially limiting their active engagement in research activities. We worked with the multisector Project Advisory Board (PAB) for the Patient Centered Outcomes Research Institute (PCORI) funded study – Innovative Care Model for Older Adults with Chronic Heart Failure (i‐COACH), to understand barriers and facilitators to virtual engagement.

Methods: The PAB started with in person meetings in March 2020. In person PAB meetings shifted to a virtual format in April 2020 using the Zoom videoconferencing platform. Members were interviewed in May 2020 regarding their experiences and satisfaction using the virtual format including facilitators and barriers to engagement. Research team members also expressed their perspectives of the virtual meeting.

Results: Nine PAB members participated in the first virtual meeting on April 22, 2020. Four PAB and three research team members were able to attend using video. Several PAB members and research team members did not have webcameras to attend via video; therefore, they attended audio‐only. Nine PAB members participated in the qualitative interview about the virtual meeting. All nine rated themselves from an eight to 10 on a scale of one to 10, with 10 being extremely comfortable with technology and one being extremely uncomfortable with technology. Though all PAB members rated themselves as being comfortable with technology, they noted that this was in part due to having to use videoconferencing platforms in other areas of their lives (i.e., work, religious services, and other community organization meetings). When asked what format they preferred for the meetings, attendees had mixed reviews, with the majority stating that they understood the need for virtual meetings, but if given the preference, they would attend in person. Research team members expressed overall satisfaction with the meeting, stating that attendees were actively engaged even though it was a virtual meeting, attendees were actively engaged.

Discussion: Virtual community advisory board meetings are an alternative to in person meetings. They are another method to ensure the community remains engaged in research especially during times when social distancing is necessary and when patient and caregiver participants are advised to not participate in in person gatherings due to their own health and safety. Community‐engaged researchers may need to provide technical assistance to those less comfortable with technology to ensure connectivity and audio/video capabilities. It is also important to note that video capabilities are an important aspect of virtual engagement. So researchers should be prepared to provide equipment (i.e. webcameras, tablets, etc.) to facilitate virtual engagement. Further, in times of high stress, PAB members may need time to destress before jumping into research activities. Researchers may find it helpful to include ice breakers or time for open‐discussion in the agendas to foster team building and engagement.

49. Expedited COVID‐19 Results through EHR Portal Enrollment in the Emergency Department

Tracy A. Walker DNP, Amy Felix CRNP, Amy Brundage CRNP, Megan Dialectos CRNP, Jessica Stewart‐Collins MSM, Joseph J. Zorc, MD

Children’s Hospital of Philadelphia

Background: The COVID‐19 pandemic lead to a substantial increase in viral testing and the related workload of communicating follow‐up results for the Emergency Department (ED) Advanced Practice Provider (APP) group. We developed a quality improvement (QI) project to increase enrollment in our EHR patient portal (MyCHOP) to allow families to receive negative COVID‐19 results immediately. Using EHR clinical decision support and other interventions, our multidisciplinary team aimed to increase the percentage of discharged patients enrolled in MyCHOP awaiting COVID‐19 results to 80% by the end of fiscal year.

Methods: Using a multidisciplinary approach, the ED Advanced Practice Provider group, Information Services, ED registration team, and ED Nursing collaborated to develop workflows to increase MyCHOP enrollment and support the immediate release of results. Five PDSA cycles were implemented; a registration track board change to display MyCHOP status, a nursing order set and provider alerts to prompt staff for non‐enrolled patients, a MyCHOP advertisement was added to the ED call menu, a text only initiative for patients/families to enroll in MyCHOP, and a message column pilot to increase staff awareness.

Results: Prior to March 16th 2020, the overall MyCHOP enrollment rate for ED patients was 37%. After implementation of the initial registration staff track‐board cycle, the MyCHOP enrollment rate increased to 48%. For patients discharged with COVID testing pending, the enrollment rate was 58% in April 2020. Adding CDS prompts to order sets and other interventions increased the rate to 79% over the subsequent year. The project is currently in sustain phase with average enrollments rate ranging from 78‐80%.

Discussion: This project streamlined the process for families to receive results immediately while reducing the follow‐up burden for APPs in the ED. The success of this project has led to additional initiatives to use MyCHOP to improve the follow‐up process of other delayed lab results.

50. Applying the “Dissemination as Dialogue” Framework to An Evidence‐Based Telebehavioral Health Network Implemented in a Rural Accountable Care Organization’s Primary Care Practices

Eve‐Lynn Nelson PhD, Robert Moser MD, Jodi Schmidt MBA, Dorothy Hughes PhD, Tiffany Pothapragada PhD, Janet Richardson Barce MBA

University of Kansas School of Medicine, University of Kansas Hospital

Background: Community‐engaged research models often fit telebehavioral health evaluation across quality improvement and research initiatives, given the multi‐system nature of the work (e.g., patients/families, rural sites/providers, behavioral specialists, community stakeholders, insurers, etc.) and the importance of community champions. Community‐engaged research models champion ongoing bidirectional communication across partners, including sharing research findings. However, closing the communication loop is often challenging due to time, money, personnel, and momentum across all partners. Coined by McDavitt et al. (2017), “dissemination as dialogue” sets the stage for community‐engaged dissemination across the course of the project, maximizing community stakeholder input at the dissemination stage in order to put findings into context as well as to continue research partnerships.

Methods: As part of a federal evidence‐based telebehavioral health initiative, the Kansas Rural Telebehavioral Health Network (PI: Moser) implemented and evaluated telebehavioral health services at 11 rural and frontier primary care sites that participate in a large rural Accountable Care Organization (ACO). The presenter will underscore the importance and process of academic‐primary care partnership around dissemination of patient quality findings and broader multi‐method research results. From February 2019 through April 2021, the Kansas Rural Telebehavioral Health Network completed approximately 1,100 encounters across 255 patients (Mean age = 48 years, 71% female).

Results: The initial quantitative and qualitative findings suggest positive outcomes, including an overall 9.9% improvement from first assessment to final assessment on the PROMIS 29, a validated self‐report assessment containing items from each of the PROMIS domains (Depression, Anxiety, Physical Function, Pain Interference, Fatigue, Sleep Disturbance, and Ability to Participate in Social Roles and Activities). Qualitative interviews have been completed with 11 rural CEOs. Lessons learned include the importance of ongoing communication about disseminating findings in order to increase organizational buy‐in and continued participation through leadership turnover. This information reinforced how the Kansas Rural Telebehavioral Health Network helped meet rural primary care site, ACO, and community behavioral health priorities and advanced shared savings goals. The presenter will provide practical information about the “telebehavioral health dissemination dialogues” between the research team and stakeholders (Cunningham‐Erves et al., 2021). The presenter will share strategies for ongoing collaboration across researchers, patients, and other stakeholders, including quickly tilting to meet COVID‐19 realities.

Discussion: The presenter will summarize how this community‐engaged research dissemination process has: 1. ensured rural primary care site engagement across the multi‐year program; 2. supported meeting ethical guidelines; 3. increased value and usability of research findings; and 4. assisted in meeting stakeholder needs, particularly around sustainability plans. This iterative process continues to promote rapport building and trust among partners and support efforts to extend collaboration to other telehealth specialties and connected health opportunities. Relevant connected health dissemination and implementation science resources will be shared, including those available through the National Consortium of Telehealth Resource Centers.

51. Bridging the Gap in Graduate Medical Education: A Comparative Analysis of Medical & Fellowship Director Experiences in Telehealth

Ragan DuBose‐Morris PhD, Christopher Pelic MD, Ryann Shealy

Medical University of South Carolina

Background: Telehealth is an important aspect of the future of medicine and was crucial to many hospitals and practices during the COVID‐19 pandemic. Although its use has increased over the years, telehealth use and knowledge has not reached its full potential. Known barriers to using telehealth include lack of training/knowledge and lack of reimbursement.

Since the inception of formal GME training for telehealth, overall utilization of telehealth has increased along with the continued need for training to prepare residents for current and future practice. MUSC is one of the few institutions in the country that had formal GME training for telehealth prior to COVID‐19, over a significant time period and with multiple programs participating. By surveying GME programs, we were able to determine the current state of training and propose adjustments of offerings to meet the rapidly evolving needs of medical directors, residents and fellows.

Methods: To meet the educational and clinical goals for GME programs, we surveyed current program and fellowship directors on the self‐assessed knowledge and confidence in practicing and supervising telehealth encounters. College of Medicine residency and fellowship program directors were originally surveyed in 2016. A longitudinal follow‐up survey was sent to all 77 Directors across 25 programs with 37 Directors completing the survey (48%) in 2021. SPSS standard procedures were applied to screen the data from the surveys. Data were compared from 2016 to 2021 with additional information about current resident/fellow training processes being analyzed. Independent sample t‐ tests were used to compare data by groups of directors. Nominal data analysis was conducted on educational modality, duration and topic area related to resident training initiatives.

Results: There is a significant difference between 2016 and 2021 in the residency directors’ comfort in demonstrating/explaining at least three telehealth tools (P = 0.043) and their ability to utilize telehealth in current or future clinical, educational or research practice (P = 0.040). Therefore between 2016 and 2021, directors gained knowledge regarding telehealth implementation. At a p<0.1 significant level, there is a statistically significant difference in the two years surveyed on the participants’ comfort in determining how telehealth increases cost efficiency (P = 0.068). This increase in telehealth knowledge is likely due largely to the surge of telehealth during the COVID‐19 Pandemic to avoid patient exposure.

Telehealth knowledge and use may continue to grow as barriers are eliminated and technology continues to develop. In both 2016 and 2021, the majority of those surveyed (72.2% and 62.2%, respectively) were concerned about reimbursement as a major barrier to telehealth. Respondents report residents were more likely to be trained through experiential clinic offerings focusing on provider‐side technologies, reimbursement, and compliance topics. Over 70% of residents provide clinical care through synchronous video visits (n = 26).

Discussion: Some programs ran into limitations with telehealth when implementing to scale during COVID‐19. With further telehealth training to increase physician and patient knowledge and telehealth education to insurance companies, these barriers could be decreased and allow patients better access to telehealth. Only one respondent in 2021 considered themselves an expert on their ability to utilize telehealth as a part of their current or future clinical, educational, or research practice. Some of the program and fellowship directors surveyed did not use telehealth in their clinical practice. The lack of respondents who believe they are an expert in telehealth shows that there is room to improve telehealth knowledge in graduate medical education. The survey helped identify potential areas of curricular reform for residency programs. Incorporating telehealth into education could help to bridge this gap in knowledge and lead to more well‐rounded physicians.

52. Using Systematic Reviews to Advance Domain Knowledge and Inform Future Research: A “How‐To” Presentation With a Telebehavioral Health Exemplar to Support Telehealth Research Training

Annaleis K. Giovanetti MA, E Zhang PhD, Ilana Engel MA, Stephanie Punt MA, Mariana Rincon Caicedo BA, Prasanna Vaduvathiriyan MLIS, Eve‐Lynn Nelson PhD, Donald Hilty MD

The University of Kansas

Background: Recent improvements in technology, increased insurance reimbursement and structural support, growing federal and regional telehealth resources, and the increase in telehealth use due to the pandemic have all contributed to growth in the number of studies examining telehealth delivery across service domains. Systematic reviews serve to reduce bias and produce high‐quality evidence that can be a resource when considering the widespread implementation of these services.

Telehealth research training in systematic reviews is needed and our presentation aims to detail the process of conducting a systematic review and how to collaborate with various personnel such as university librarians and international subject experts to improve methodological rigor. We will leverage process and data from our team’s recent systematic review of telebehavioral health in youth as an example to highlight considerations and challenges when evaluating efficacy, implementation, and other areas of interest.

Methods: Given the large number of review types used to analyze published literature (Grant & Booth, 2009), we will first briefly outline other review types to consider, when to conduct a systematic review, and how to ensure methodological rigor in review design. The presentation will discuss the use of the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines, the evidence‐based standard for systematic reviews, to plan and conduct reviews as well as the option to preregister designs using the International Prospective Register of Systematic Review system (PROSPERO). Considerations and choice points for study selection will be covered, including determining scope of topic, search terms, inclusion and exclusion criteria, database selection, and data extraction and screening (Relevo & Balshem, 2011). Our challenges associated with developing the telebehavioral health research question, using a two‐reviewer system, and methodological rigor will be discussed. Given the heterogeneous definitions of the term “telehealth”, there will also be a focus on identifying the specific modality of interest and how telehealth services may have changed due to the pandemic.

Results: Although the efficacy of telehealth‐based services in mental and behavioral health has had growing support for years, implementation and use was slow prior to the COVID‐19 pandemic. The exemplary systematic review of these services in youth used subject experts and librarian expertise to help execute a high‐quality systematic review across this large topic area. This review demonstrates the utility of systematic reviews in providing rigorous evaluations of the research, which is increasingly vital as the use of telehealth‐based services increases dramatically. For the exemplar pediatric telebehavioral health review, reviewers screened 3450 titles and abstracts and reviewed over 340 full‐text articles from 2000 to 2021 from the PubMed, PsycInfo, and EMBASE databases as of August 2021. Decisions related to inclusion and exclusion criteria were discussed at regular meetings and elevated to subject matter experts if consensus could not be reached. Discussion of how a shared citation management tool facilitated the review and other helpful resources will also be shared. Effective strategies to engage students, collaborators, and team members remotely will also be discussed to promote collaboration across institutions.

Discussion: Systematic reviews serve as important resources for synthesizing existing research and providing best‐practice recommendations for future telehealth research. This is an increasingly important method to understand and be able to execute at a high‐level as studies of telehealth are likely to grow substantially following the pandemic. We will conclude with vetted resources related to building and enhancing these important telehealth research skills in order to help investigate effectiveness and implementation outcomes across telehealth research that may suffer from practical constraints. Evaluating the quality of available evidence and the dissemination of this work allows both providers and consumers to make more informed decisions regarding telehealth use and promotes informed policy decisions. This “how‐to” style presentation with examples from a review of telebehavioral health for youth illustrates the impact of this method and highlights associated challenges and practical benefits.

53. Quality indices and outcomes of a neonatology telerounding program: single center experience during COVID pandemic

Elena Bosque PhD,¹ Christine Cooper DNP,² Rossella Mastroianni MD,² Shilpi Chabra MD,² Julie Campbell NNP, Jose Perez MD,² Rachel Umoren MBBCh²

¹Seattle Children’s Hospital, ²University of Washington School of Medicine, ³University of Washington Northwest Hospital

Background: Telehealth is increasingly used in neonatal intensive care, especially during the COVID pandemic. There is emerging literature regarding the benefits of telehealth for neonatal patients at various levels of neonatal care facilities including on neonatal transport. The feasibility of a hybrid telehealth model has been demonstrated for application in Level II special care nurseries (SCN), with the neonatologist rounding in person or by telehealth with neonatal nurse practitioners (NNPs) on site. However, there are no known descriptions of the outcomes of telerounding in SCNs with only neonatal advanced practice providers (APP), including neonatal physician assistants, on site and few recommendations for others to develop similar programs. The purpose of this study was to describe outcomes, including critical quality indices for remote neonatal telerounding in Level II SCNs, provider satisfaction, perceptions of parental receptiveness to neonatal telerounding.

Methods: This is a mixed methods study on qualitative and quantitative outcomes of telerounding in a Level II SCN within a regional neonatology program in the Pacific Northwest. Rounding occurred at the bedside using a specialized telehealth cart with a paired Bluetooth stethoscope and parents were often present. The frequency of parent presence and neonatologists’ rating of the value of video telemedicine and technical difficulties, as well as their perceptions of parent impressions were collected prospectively by survey after each telerounding experience. Additionally, a cross‐sectional survey of APPs was administered in March 2021. De‐identified patient demographics and outcomes (gestational age, birth weight, length of stay etc.) were obtained through retrospective chart review from May 1, 2019‐Feb 28, 2021 and compared between two epochs: pre‐COVID‐19 pandemic from May 1, 2019‐Feb 28, 2020 and post‐onset of COVID‐19 pandemic from March 1, 2020‐Feb 28, 2021. Patient demographic and outcome data were summarized and described as median and interquartile range (IQR). Perceptions of parent impressions were grouped by common responses, summarized, and described. Survey results were analyzed via content analysis.

Results: 12 neonatologists and 23 APPs participated in the program; 7 neonatologists and 15 APPs completed the survey (63% response rate). A total of 258 patients were admitted with median gestational age of 37.4 weeks, birthweight, and length of stay, 2949g and 3.7 days respectively. 43.8% were female, 51.3% were born after vaginal delivery, 17% had antenatal steroids, 88% breastfed at discharge. There were no significant differences in patients’ characteristics and outcomes pre‐ and post‐onset of the COVID‐19 pandemic.

The providers indicated overall satisfaction, with the lowest score regarding received training. The common themes concerning what was “least liked” were “need for goodness of fit” and “technical difficulties”, while “most liked” were “another set of eyes”, “support of the customer”, and “opportunities for use”. Responses about parental receptiveness were positive with families being appreciative, comfortable, and engaged. Benefits included the ability for consultation with a neonatologist and a second opinion. The “customer” included the family, APP providers and nurses who liked having the physician involved in the care, and the institution, supporting the system that allowed the neonatologists’ remote presence.

Discussion: Telerounding in a Level II SCN, with a team of physician and APP providers, resulted in stable provision of care with no differences in clinical outcome during the COVID‐19 pandemic and provided opportunities for assessment and communication. This is one of few descriptions of neonatal APP telerounding in a SCN setting and the potential impact of the COVID‐19 pandemic. We found that there was a consensus of satisfaction among providers, nurses, and parents. The negative responses related to logistical, technical, and training issues. Other beneficial uses for telemedicine in a Level II SCN include support for unexpected outcomes in the delivery room resuscitation. In conclusion, telerounding in neonatology is an effective and feasible method of providing neonatal care, especially in remote, rural areas where there is increased need of sophisticated medical decision making in the care of these tiny fragile babies.

54. An Empirical Investigation of the Impact of COVID‐19 in Telemedicine Effectiveness

Rachel Umoren MBBCh,¹ Naveen Kumar PhD,² Liangfei Qiu PhD,³ Subodha Kumar PhD,⁴ Taylor Sawyer DO¹

¹University of Washington School of Medicine, ²University of Oklahoma, ³University of Florida, ⁴Temple University

Background: The COVID‐19 pandemic created a significant disruption in traditional office visits and telemedicine was used by many providers to address this disruption (Rai 2020). The prior literature on telemedicine has not differentiated between telemedicine adoption and usage (Bavafa et al. 2018). We explored how this disruption of medical office visits affected telemedicine adoption and usage. To systematically bridge key research gaps, we begin by asking our research questions.

• (i) How did the COVID‐19 pandemic affect (a) telemedicine adoption of new patients and (b) telemedicine usage of existing patients?

• (ii) How do physicians’ experiences moderate the impact of the COVID‐19 pandemic on (a) telemedicine adoption of new patients and (b) telemedicine usage of existing patients?

• (iii) How does patient age moderate the impact of the COVID‐19 pandemic on (a) telemedicine adoption of new patients and (b) telemedicine usage of existing patients?

Methods: A total of 155,263 patient records were reviewed from de‐identified datasets obtained from a highly ranked medical center in the Pacific Northwest region of the United States: An outpatient encounter dataset with individual patient‐level information on visit day, time, diagnosis, encounter type (telemedicine or in‐person), patient demographics (age, gender, location, etc.), and attending physician demographics (degree, graduation year, and gender). We use a patient’s encounter (telehealth or in‐person, 35 weeks before and after the COVID‐19 outbreak) with a physician as our unit of analysis. We used the number of confirmed COVID‐19 cases in each county of Washington state over several weeks as a control variable in this study. We obtained this data from the Washington State Department of Health and supplemented it with the outpatient encounter and the clinical diagnosis datasets. We used the county name and the week information to merge this data with the outpatient encounter and diagnosis datasets. We examined the data for each research question using the fixed‐effects regression model. We discussed the results of the fixed‐effects regression model in the Results section.

Results: Our results showed that after COVID‐19, the telemedicine visit percentage increased by 12% (p<0.01). Going one step further, using the fixed‐effects estimation model, our nuanced statistical analysis and results showed that the COVID‐19 pandemic increased telemedicine adoption more (16%, p<0.05) than usage (9%, p<0.01) by previous users. Since previous telemedicine leads to more in‐person medical visits later, a patient who had experienced telemedicine before the COVID‐19 pandemic is more likely to physically visit doctors’ offices and is less likely to use telemedicine than a patient who had not experienced telemedicine before the COVID‐19 pandemic. In addition, we observed that the positive impact of COVID‐19 on the continued telemedicine usage appeared stronger with more experienced doctors (8%, p<0.01). From the patient age perspective, we observed a positive effect (p<0.01) for telemedicine usage in the previous user/adopter case and a negative effect (p<0.01) for telemedicine adoption in the new adopter sample.

Discussion: We observed that after COVID‐19, patients are more likely to adopt telemedicine. The COVID‐19 pandemic affected telemedicine adoption of new patients and telemedicine usage of existing patients differently. A possible reason is that telemedicine leads to increased interactions between doctors and patients and can cause more in‐person medical visits due to the “gateway” effect (Bavafa et al. 2018). These results highlight that policymakers and healthcare practitioners should pay close attention to the difference between telemedicine adoption of new patients and telemedicine usage of existing patients. In addition, the disruptive effect of the COVID‐19 pandemic on telemedicine is moderated by doctor experience and patient age. Third, our evaluation of the impact of the COVID‐19 pandemic on the effectiveness of telemedicine helps policymakers and healthcare practitioners understand the effectiveness of telemedicine compared to physical visits.

55. Leveraging Existing Resources to Provide Telehealth Education for Nurse Practitioner Students and Rural Preceptors

Karen Nellis BSN,¹ Sarah Rhoads PhD,¹ Dee Blakney DNP,² Jackie Sharp DNP,¹ Laura Reed DNP,¹ Alan Faulkner BA,³ Bobby Bellflower DNSc¹

¹University of Tennessee Health Science Center, ²Whiteville Family Medical Clinic, ³Faulkner Consulting

Background: At the height of the stay at home orders of the COVID‐19 pandemic, many outpatient clinics saw a drastic decrease in patient clinic visits. Many small rural nurse practitioner‐owned clinics struggled to care for patients. This decrease in visits impacted not only patients and health care providers but also students requiring clinical hours through one‐on‐one precepting for graduation. Some students saw the need to find additional clinical experiences or possibly delay graduation. The University of Tennessee Health Science Center College of Nursing had current funding through the Health Resources and Services Administration (HRSA) Advanced Nursing Education Workforce to support primary care nurse practitioner students and rural nurse practitioner preceptors. Faculty leveraged this funding to train preceptors and students regarding best practices in telehealth.

Methods: Faculty and staff traveled on‐site to partnering underserved and rural clinics to conduct telehealth needs assessments. Based on each clinic’s needs assessment, telehealth equipment was placed at each site to aid in nurse practitioner students’ education and clinical experiences of nurse practitioner students. The on‐site, hands‐on training was provided by experienced telehealth trainers using the equipment placed at their clinic. In addition, telehealth simulations were integrated into the family and psychiatric mental health nurse practitioner programs.

Results: Telehealth experts conducted needs assessments at two rural clinics and one urban clinic serving a medically underserved population. Each site varied on the type of equipment needed (iPads/tablets, laptops, web cameras, telehealth cart with peripherals, stethoscopes, etc.) to support student learning and care for their unique patient population. Faculty and staff visited each site to deliver equipment, provide ongoing technical and programmatic support, conduct student site visits, and conduct periodic check‐ins to assess for issues. Students received hands‐on telehealth training and clinical experiences, such as preceptor‐observed care for patients via telehealth or assisting patients to connect with providers at a remote location. Telehealth educational training materials were developed and placed on the learning management system CE Now (www.CENow.uthsc.edu). In addition, a satellite telehealth training center is scheduled to open on campus in Fall 2021 in collaboration with the HRSA‐funded South Central Telehealth Resource Center. Telehealth visits are currently being conducted at all three partnering clinical sites.

Discussion: Small rural, independently owned clinics did not have the infrastructure that larger health systems had to purchase telehealth equipment, deliver telehealth training for providers, troubleshoot connectivity or other information technology (IT) issues, assist patients in connecting for virtual visits as well as assess telehealth needs. Ongoing programmatic and technical support was essential for these rural providers to become adept at providing virtual care to patients and simultaneously mentoring nurse practitioner students during their clinical rotations. Adjustment of programmatic deliverables in coordination with funding program officers to include telehealth training improved nurse practitioner student educational experiences and provided the needed programmatic and IT assistance for small independently owned clinics.

58. A Regional Pediatric Telehealth Consortium Lays the Foundation for a Pediatric Virtual Hospital

Benjamin Parrish BS,¹ Ricardo Munoz MD,^1,2 Natasha Shur MD,^1,2 Joelle Simpson MD,^1,2 Donya Forohar,³ Cyrus E. Sabouriane¹ Craig Sable MD,^1,2 Alejandro Lopez‐Magallon MD,^1,2 Deena Berkowitz MD,^1,2 Clarence Williams BS,¹ Shireen Atabaki MD^1,2

¹National Hospital, ²The George Washington University School of Medicine and Health Sciences, ³University of Maryland

Background: Our health system established a regional pediatric telehealth consortium (RPTC) in response to the COVID‐19 pandemic. The program enabled; expansion of an established telehealth platform to support 15 urban, suburban and rural healthcare sites in a tri‐state region; continuity of care for patients while limiting community spread of COVID 19; and remote care to expand region‐wide surge capacity. Virtual interpreter service platforms and tablets to maintain family presence for inpatients were provided. Ambulatory pediatric specialty care transitioned to a telemedicine first model. Direct to consumer telemedicine follow‐up clinics were established post emergency department and inpatient discharge. The program was supported in part by the Federal Communications Commission’s COVID 19 Telehealth Program.

Our objectives were first to establish a pediatric telehealth consortium to coordinate and provide care via network technology and broaden regional access to pediatric specialty service.

Methods: This was a mixed methods observational study with data from the post‐COVID period between March 1, 2020 to August 31, 2020, from parent and provider surveys, financial and electronic health record and direct to consumer telemedicine platform data. A dashboard was developed to track telemedicine program assets, deployment, utilization, patient and provider characteristics and geographic distribution of visits. Provider surveys were carried out to gain insight into the telemedicine program from users.

Results: Between March 1 to August 31, 2020, our health system provided 62,864 DTC telemedicine visits, with 558 providers. Telemedicine patients were 51.7% female, insurance was 45.7% public, 34% private and 8.7% federal. In response to the COVID‐19 pandemic, telemedicine patient volume increased 284% across all specialties and subspecialties over the course of March and April 2020. Additionally, the number of providers of direct to consumer telemedicine increased by 38% between February (400 providers pre‐COVID) and March 2020 (554 providers post‐COVID). The top 10 direct to consumer (DTC) visit diagnoses constituted 21.9% of diagnoses and included behavioral health diagnoses, routine child health exams with and without abnormal findings, diabetes and asthma.

Behavioral health related diagnoses occurred with higher frequency among all patients (6/10) compared to publicly insured patients (2/10). Approximately 40% of DTC dermatology visits were provided to patients in underserved zip‐codes. Emergency department, critical care, inpatient and DTC telemedicine was established to provide access to specialized expertise across the 15 regional partner sites and throughout the community.

Discussion: Establishment of the RPTC supported our health system in, a shift to a telemedicine first model, treatment of COVID‐19 and provision of specialty pediatric care to patients via telehealth across a tri‐ state region. As we expand telehealth services, we are aware of addressing digital divides, where access to care could be limited by families lacking equipment needed for video visits: smartphones, computers and a reliable internet connection. Our program has been able to serve underserved patients and overcome potential disparity in care based on SES and access to technology. As we grow a virtual hospital, the dashboard provides the framework to instantly view information in real time in efforts to consider distribution of care to underrepresented zip‐codes, analyzing patterns, and acutely responding to identified barriers. This “virtual hospital” model has been designed with technological pathways that identify healthcare disparities, address needs in real‐time, and broaden our reach.

59. The Birth of Telelactation during the COVID Crisis

Kara Benneche MSN, Linda Stopsky BSN

Northwell Health

Background: By age 6 months only one half of all infants born in the US receive any breast milk and only 22% are breastfed exclusively as recommended by the American Academy of Pediatrics. Majority of women stop breastfeeding earlier than desired due to difficulty with breastfeeding. The Covid‐19 pandemic led to shortened hospital stays for post‐partum women and their babies thus limiting traditional in hospital education. In March of 2020, Northwell Health created a telelactation service as an innovative way to support moms through the breastfeeding process. In response to the Covid‐19 crisis, post‐partum mothers and infants were being discharged early after giving birth. With little resources available and limited education provided during their hospital stay, many moms had little to no support related to lactation services. In addition, many patients did not have family resources to support their discharge due to quarantine recommendations. Northwell Health identified the need to support new moms and initiated a telelactation program at two of our facilities. Telelactation consults were initiated using audio‐visual technology and professional lactation consultants to answer questions related to the breastfeeding process, resolve existing breastfeeding issues, and provide support to mothers and fathers in the comfort of their own home.

The goal of the telelactation program at Northwell Health is to provide remote lactation consultations via audio visual technology thereby supporting new parents related to the breastfeeding process and provide breastfeeding support to mothers in the comfort of their own homes. Studies have shown that telelactation offers a service that rivals in‐person care and is often more convenient for the mother, less costly, and encourages self‐efficacy.

Methods: During the Covid 19 crisis, a need for telelactation services was identified at Northwell Health. The program started with one registered nurse who is also an International Board Certified Lactation Consultant (IBCLC). A telelactation script and workflows were created. The telelactation RN received education related to telehealth use, privacy expectations, camera presence and troubleshooting technical issues. Patient referrals were communicated via email from hospital based lactation consultants at two Northwell sites. The telelacataion RN emailed referred moms to set up 1 hour appointments within 72 hours of discharge. In the initial phase of the telelactation program (March 2020‐October 2020), 335 telelactation consultations were successfully completed.

Results: During the Covid 19 crisis, a need for telelactation services was identified at Northwell Health. The program started with one registered nurse who is also an International Board Certified Lactation Consultant (IBCLC). A telelactation script and workflows were created. The telelactation RN received education related to telehealth use, privacy expectations, camera presence and troubleshooting technical issues. Patient referrals were communicated via email from hospital based lactation consultants at two Northwell sites. The telelacataion RN emailed referred moms to set up 1 hour appointments within 72 hours of discharge. In the initial phase of the telelactation program (March 2020‐October 2020), 335 telelactation consultations were successfully completed.

Discussion: Through the implementation of a telelactation program, Northwell Health aims to introduce innovative strategies to provide patient education and complete the continuum of care for parents and their babies. During the Covid 19 crisis, in some cases, Northwell Health offered the only available breastfeeding support and education to many moms. Northwell Health plans to expand the telelacatation program to include additional sites as well as offer exclusive, concierge telelactation service to Northwell Health employees. Through our poster presentation the learner will be able to: explain that the use of audio‐visual technology is an effective method of providing education, assessment and support to breastfeeding mothers, describe the benefits of improving access to lactation services through the use of audio‐visual technology, and learn how to incorporate a telelactation program at their own facility.

LB01. Telehealth Utilization During the COVID‐19 Pandemic: A Preliminary Selective Review

Amelia Harju MPH, Jonathan Neufeld PhD

University of Minnesota

Background: The impact of the COVID‐19 pandemic on the delivery of health care services in the United States was rapid and profound, reducing in‐person visit volume and fueling a corresponding explosion in demand for services that could be delivered remotely to patients outside the clinic setting. In response to this rise in demand, several temporary policies were enacted at state and federal levels to allow greater flexibility in delivering health care services via telehealth during the public health emergency (PHE).

Several researchers have published studies using large claims datasets to explore this surge from a variety of perspectives. We undertook this review to begin to synthesize findings available from the large‐scale studies published thus far, and also to lay some methodological groundwork for the interpretation of these and any future findings related to telehealth utilization patterns during the pandemic.

Methods: This collection of studies was constructed using Ovid MEDLINE and Google Scholar. To be included in this review, the studies must be original research, include data from 2020‐2021, have a study population that is exclusively in the United States, and include an analysis of telehealth electronic health records or claims data across multiple payers and health systems.

The Ovid MEDLINE search produced 137 articles, of which three were selected for the primary analysis for this review and 11 were included for the peripheral analysis. The Google Scholar search produced 5,690 articles. Articles from the first 10 pages were considered for inclusion, of which seven were selected for the primary analysis of this review and 17 were included for the peripheral analysis. This review synthesizes information from a total of ten articles that analyzed large‐scale, multi‐payer, multi‐health system datasets to examine patterns in telehealth utilization during the COVID‐19 pandemic. This review also briefly summarizes information from an additional 29 articles that examined telehealth use during the pandemic, albeit not from multi‐payer, multi‐health system datasets.

Results: Across the board, in‐person visits declined during the pandemic while telehealth visits increased. Several reports indicate that primary care visits dropped by as much as 50% early in the pandemic, then rebounded over the course of the rest of the year. At its peak utilization, telehealth accounted for roughly 30‐50% of visits in many datasets. Most reports found that audio‐only telehealth was used more often than video visits.

Several reports found that telehealth utilization varied across geographic regions, but results conflicted regarding which regions utilized more or less telehealth services. Telehealth use also varied widely across specialties and diagnoses, with behavioral health having the highest telehealth utilization. Generally, the more telehealth was utilized, the smaller the observed decrease in visit volume during the pandemic.

Overall, telehealth utilization was higher among patients who live in urban areas, areas with greater broadband availability, and areas with higher pre‐pandemic telehealth utilization. Telehealth utilization was also higher among patients who were White, spoke English as their first language, were middle‐aged, and had health insurance, higher incomes, and greater disease burdens.

Discussion: The majority of telehealth visits that occurred during the pandemic would not have been reimbursable without the telehealth‐related policy changes that took place during the PHE. These changes allowed for telehealth services to partially offset the decline in in‐person visits, reduce the risk of COVID‐19 transmission, preserve PPE, and maintain access to necessary health care services. Audio telehealth appears to often function as a fall‐back when video isn’t possible. It also seems likely that the variability in telehealth utilization across geographic regions is more accurately attributed to telehealth policies. Most studies examining disparities in telehealth utilization did not compare differences in patient demographics from before and during the pandemic, and the one study that did found no significant differences. Therefore, disparities in telehealth use may be a characteristic of health care and society rather than of telehealth specifically.

LB02. Leveraging Telehealth for Physical and Psychological Assessment and Intervention at Pro Bono Community Clinic in California Central Valley during COVID 19

Dr. Bryan Kwon DPT, Katiria Alexandra Penson SPT

California State University

Background: On March 11, 2020 the World Health Organization (WHO) declared coronavirus (COVID 19) a worldwide pandemic. Since then, it has affected 190 million people worldwide, specifically in the United States the number of infections from this deadly outbreak amounts to 34 million people. In California, 4.35 million confirmed COVID‐19 cases have resulted in 65,228 deaths. The unprecedented scale of this pandemic rapidly changed the landscape of telehealth. It enforced swift and rapid adoption of telehealth to provide medical care. Thereby, significantly impacting all aspects of healthcare delivery and education. Telehealth is a valuable approach because it increases access to care particularly among populations that need access the most especially in rural communities. The hands‐on pro bono community clinic which was transitioned into an innovative telehealth clinic for Physical and Psychological Assessment with neurological disorders.

Methods: This case report utilizes a client who was a 79‐year‐old female who reported to the telehealth clinic for help with gait and balance. Since the onset of the pandemic, she experienced deterioration in her balance, a difference in her gait, and a decline in her ability to rise from a chair and toilet. Her primary goal was to stand from a chair and toilet without the use of an arm rest or grab bars. Under the supervision of a licensed clinical instructor (CI), two SPTs conducted an evaluation, examination, assessment, and created a plan of care to address one client’s functional limitations. The individual telehealth session comprised a 45 min therapy session followed by a 15‐minute debriefing between the CIs and SPTs. The student physical therapists also used a mhealth platform called Health in Motion (HIM) developed by Blue Marble Health. The telehealth education model for physical therapy education was based on Four P’s of Telehealth Framework that consisted of four domains (1) planning, (2) preparing (3) providing (4) performance evaluation.

Results: The client underwent physical and psychological Assessment in the form of objective examination, satisfaction surveys and the Telehealth Usability Questionnaire (TUQ). The patient was subjected through a comprehensive objective examination that consisted of depression screen, clinical and functional outcome measures. The two SPTs who evaluated the client completed an assessment and discussed clinical reasoning and clinical decision making with their CI. The client’s signs and symptoms were consistent with poor somatosensory function, reduced vestibular function, poor safety awareness along with generalized deconditioning. The deficits contributed to decreased strength, balance loss, and increased fall risk. The patient was subjected through a comprehensive balance assessment and objective examination (refer table 2) that consisted of various clinical and functional outcome measures including the Timed Up and Go (TUG) test, modified mCTSIB, Berg Balance Test (BBT), 4‐Stage Balance Test (4SBT), Heel Standing, and the 30 Second Sit to Stand Test (30 STST). The client logged in 13 hours and 21 min on the HIM app thereby demonstrating using the app for a home exercise program.

Discussion: In academia, the pandemic enforced the historic migration of clinical coursework to remote environments and has offered a once‐in‐a‐lifetime opportunity to reimagine and reframe the delivery of academic and clinical coursework and visualize the pros and cons of telehealth. Continuation of care in physical therapy is essential to meaningful improvements in patient outcomes. In this regard, telehealth provides opportunities for the patients to extend rehab efforts when physical distance and mode of transportation poses a barrier in obtaining care. Telehealth is a feasible, safe, and well‐ received alternative in student clinical education and neurological physical therapy needs. Reimagining, restructuring, and adopting innovative ways to teach clinical education is needed as we move in the era of hybrid work environments.

LB03. Estimating time‐driven cost differences for a telerehabilitation versus outpatient rehabilitation session

Corey Morrow PhD‐C, Kit Simpson DrPH, Michelle Woodbury PhD

Medical University of South Carolina

Background: The cost of stroke care in the US is the highest in the world. Additionally, access to stroke rehabilitation is limited for certain demographic groups including rural and low‐income stroke survivors. The odds of receiving rehabilitation therapy are 1.2 times greater for urban (>1000 civilians per square mile) than rural stroke survivors (between urban and highly rural), with 45% decreased odds for highly rural (population density <7 per square mile) stroke survivors.

Telerehabilitation is an emerging service delivery method that may improve both cost and access for stroke survivors. The objective of this study is to estimate the marginal cost differences between a telerehabilitation versus outpatient session for stroke survivors.

Methods: This study used a modified time driven activity‐based costing approach which included: 1) structured and recorded interviews with two occupational therapists familiar with outpatient therapy and two therapists familiar with telerehabilitation, 2) collection of standard wages for providers, 3) collection of appropriate billing codes for stroke survivors, 4) the creation of an iterative flowchart of both an outpatient and telerehabilitation session care delivery process, and 5) sensitivity analyses to account for potential cost uncertainties. This approach is more appropriate than other costing methods as it describes the individual costs necessary to provide a session as opposed to traditional lump sum approaches. Total costs for both flowcharts were compared to define cost differences between outpatient and telerehabilitation services.

Results: Early results indicate telerehabilitation may be a less costly method of service delivery for an occupational therapy session. These results are heavily influenced by a reduction in patient transportation time. Final results are still being validated by clinician experts.

Discussion: These results give supporting evidence to translate telerehabilitation into standard practice and improve reimbursement. This study also identified possible areas to improve the delivery of care including the addition of clerical staff for appointment scheduling and coordination. Telerehabilitation has the opportunity to reduce costs, improve access for rural patients, and improve functional outcomes for more stroke survivors.

LB04. Evolution of Telemedicine Use Among Medicare Beneficiaries

Zhongran (Adrian) Niu, Jonathan Neufeld

University of Minnesota, Great Plains Telehealth Resource and Assistance Center

Background: Telemedicine is growing as a popular, safe, and convenient way to see a medical provider during the COVID‐19 pandemic. Ongoing monitoring and public reporting of telemedicine use among Medicare providers and beneficiaries are essential. Having a holistic understanding of telemedicine utilization will help inform providers, payers, and the public regarding the major telemedicine transitions in our system of care, allowing them to adjust resources to better meet the needs of patients. The purpose of this research is to provide accessible summaries of telehealth utilization for the public and telehealth providers. 1. Evaluate the use of telehealth services between 2016 ‐ 2019. 2. Describe and visualize patterns of telehealth service use with interactive maps and charts. 3. Describe the relative distribution of telehealth services across the United States stratified by services type, clinician specialty, and location (state).

Methods: The telehealth visualization tool was built utilizing the CMS Public Use Files (PUF) by extracting telehealth data with SQL and visualize it on Google Data Studio. We extracted the 2016 ‐ 2019 Physician/Supplier Procedure Summary (PSPS) PUF to provide a detailed analysis of telehealth utilization by joining appropriate HCPCS codes, place of service codes, and modifiers that indicate telehealth services. Claim data were elaborated using existing tables that provide descriptors for service, carrier, and provider type codes. Service data were summarized by state and specialty to show changes in utilization patterns over time and by geography. The Google Data Studio was used to report and disseminate the result, and to visualize how service delivery patterns evolved before the pandemic.

Results: The tool allows users to search by state, provider type, and CPT code and provides both map‐based and graphical visualizations. By joining the HCPCS code and specialty code with the existing description table, we find that Vermont, Nevada, and Kansa are the top 3 states for 2019 telehealth allowed services (Per 10,000 Enrollees). Among the HCPCS codes, Established Patient Office Visit, Psychotherapy, and Inpatient Telehealth Follow‐up are the top 3 services utilized via telehealth. The tool can allow combination/filter of any service types, CPT codes, and states to visualize the telehealth utilization among all 50 states including Virgin Islands, Puerto Rico, American Samoa, Micronesia, Guam, Marshall Islands, Northern Marianas, and Palau, etc. If a more recent CMS PSPS data is published, it can be added to the database of the tool to reflect new updates of telehealth utilization.

Discussion: Prior to the Covid‐19 pandemic, telehealth was growing at a steady rate under Medicare, but there were wide variations among states. An accessible visualization tool may help researchers and others better understand the differences in telehealth utilization among states and the factors that may be driving these differences. CMS also provides the limited data set (LDS), which contains beneficiary level health information but excludes specified direct identifiers ruled by HIPAA. The tool has the potential to provide a more detailed analysis of telehealth utilization in a more finely grained timely (weeks and months, rather than years). It would also allow geographical visualization of telehealth activity at the zip code level. For the future version, we intend to develop summaries of telemedicine services by week, state, service code, and provider type, suppressing any cells that fall below 11 cases.

LB06. A Retrospective Look at the Long‐Term Effects of Remote Patient Monitoring

Tearsanee Carlisle Davis DNP, Yunxi Zhang PhD, Ashley S. Allen MSN

UMMC Center for Telehealth

Background: Remote patient monitoring (RPM) has demonstrated value as a tool to aid patients in the management of their chronic illness in the home. Although the Mississippi Diabetes Telehealth Network Study (MSDTNS) was successful in reducing HbgA1c levels for patients participating in RPM in the Mississippi Delta, it remains unclear the long‐term effect of RPM on patients and how to support patients to maintain the treatment effect after discharge. This study aimed to look at the health status of individuals who were participants in the Mississippi Diabetes Telehealth Network Study (MDTNS) in North Sunflower County Mississippi to see if the outcomes were sustained once the remote patient monitoring program ended for them.

Methods: A retrospective review of medical records of patients who completed all phases of the MDTNS from 2014 to 2016 was performed over a period of 6 months. Data collected included HbgA1c values, demographics, and changes in social determinants of health. For patients who did not have current labs, the A1c was drawn at the clinic site.

Descriptive statistics, mean and standard deviation for continuous variables, frequency and percentage for categorical variables, were used to summarize baseline and last HbgA1c measures, the outcome variable and baseline patient characteristics. Wilcoxon signed rank test was conducted to examine the change of the HbgA1c from baseline to the end of the study. Linear regression models were performed on available cases to evaluate the effect of each socio‐economic variable on the change of HbgA1c, adjusting baseline measures of HbgA1c. If the significance was detected in regression at 0.1 level, we further evaluated the change of HbgA1c for the socio‐economic variable. Here, we conducted Wilcoxon signed rank test again to examine the change of HbgA1c for each race group.

Results: Thirty‐one out of 115 completers from the original study consented to participate in this study. The remaining 84 completers were lost to care and could not be found to offer participation in this follow‐up study. The HbgA1c was recorded at baseline for 30 participants with a mean of 7.76 (sd = 1.26). At the end of the study, the HbgA1c was increased by 0.27 (sd = 1.77) on average from the baseline. No significant change was detected at 0.05 level through the Wilcoxon signed rank test (p‐value = 0.33). Race was slightly associated with the change of HbgA1c (p‐value = 0.07). Compared to Caucasians, African Americans are expected to have a 1.16 (sd = 0.62, 95% CI = (‐0.11, 2.42)) higher increase in the outcome at the end. Other socio‐economic variables do not show a significant effect on the change of HbgA1c.

The change of HbgA1c was further investigated for each race group. African Americans had an average increase of 0.72 (sd = 1.77, p‐value = 0.01) in the HbgA1c, while Caucasians had an average decrease of 0.64 (sd = 1.45, p‐value = 0.20) in the HbgA1c.

After multiple imputation on baseline HbgA1c, age, and married (yes/no), the results of regression analysis show that African Americans have a 1.19 higher increase in HbgA1c.

Discussion: This study has a small sample size and missing values with the retrospective nature, which was flawed by the lost follow‐up two years after the original study completion. This limitation reduces the statistical power of detecting significance. As such, to evaluate the long‐term effect of RPM more comprehensively, it would be desirable that future prospective experiments design the study with a follow‐up period after removing interventions. In rural and underserved areas, losing patients to follow‐up is not uncommon. Remote patient monitoring provides ongoing support for patients at highest risk for poor health outcomes. Although all patients who completed the original study had good outcomes, the findings of this follow‐up study indicate that there is a high probability patient outcomes will suffer without some form of continued engagement. To evaluate the long term effects of remote monitoring, a larger sample size must be followed proactively over time.

LB07. Measuring EHR data quality in Type 2 Diabetes Mellitus (T2DM) Care

Kevin Wiley MPH, Eneida Mendonca MD, Justin Blackburn PhD, Joshua R. Vest PhD

Indiana University

Background: Electronic health records (EHR) are a common source of secondary data for reuse in clinical and health services research. Nevertheless, such data may not be well‐suited for use due to numerous data quality challenges. EHR data are variable within and across systems due to manual entry, lags between patient visits and documentation, differential documentation guidelines, and changes in standards. In particular, discordant and missing data are common error types in larger healthcare datasets where coded and narrative text are voluminous.

The research highlighting the above quality challenges and concerns were largely identified from EHR data generated during a traditional in‐person visit between a provider and a patient. However, remote or telehealth visits have steadily increased over the past 20 years and became even more common in response to the COVID‐19 pandemic. These different care modalities may have further affected EHR data quality.

Methods: For this study, we used EHR data from an enterprise data warehouse for 26,104 patients (287,644 encounters) managing T2DM aged ≥18 years who were seen between 2016‐2020 at two health systems in central Indiana. We measured timeliness among telehealth and office‐based outpatient visits as the total number of days between the encounter date and T2DM measurements measurement for each patient encounter. Completeness was measured longitudinally at the patient‐ encounter level as the number of complete demographic and T2DM measurements divided by the total relevant demographic and T2DM data elements from the first through the final patient visit.

Computed ratios and scores closer to 1 indicated that data documented as a result of either delivery format were more timely or complete. We quantified data quality differences among visit types using means, standard deviations, standard errors, and interquartile ranges. Pearson correlation coefficients described relationships between data quality measures and visit type.

Results: The majority of the clinical visits were for patients who were identified as African American (55%) and female (63.1%). The mean patient age in the sample was 53 years. We found notable variations in timeliness measures among telehealth and office‐based visits for each T2DM measurement. Average timeliness remained flat prior to March 2020, at which point total days between the documented encounter and lab measurement as a result of office‐based compared to telehealth visits increased substantially from an average of ∼7 days to ∼72 days which may be attributed to COVID‐19‐related transitions in care delivery formats.

However, results indicate that documentation as a result of an office‐based visit was more timely and completeness than documented telehealth visits for body mass index (15.6 vs. 69.1 days; p<0.01), body weight (15.7 vs. 70.4 days; p<0.01), blood pressure (12 vs. 65.1 days; p<0.01), HbA1c (397 vs. 432 days; p<0.001), Cholesterol (312 vs. 565 days; p<0.001), Serum Creatinine (95.8 vs. 120 days; p<0.001) and smoking status (16 vs. 30.3 days; p<0.01). We found that the mean completeness scores among data resulting from office‐based and telehealth visits for all relevant data were 0.97 (SD = 0.06) vs. 0.96 (SD = 0.09).

Discussion: EHR patient data resulting from office‐based visits for T2DM were more timely and complete than data documented as a result of or during a telehealth visit. These results held for data documented as a result of office‐based visits for relevant T2DM indicators and measurements. However, we were able to identify minor improvements in timeliness when we stratified analysis for telehealth data.

While improvements to the timeliness of data capture for patients managing T2DM were minor among telehealth visits, prior research found that the use of telehealth hardware (e.g., tablets) may substantially improve the timeliness of data entry. Generally, these results did not hold when telehealth visits were compared to office‐based visits. Importantly, timeliness for data documented as a result of an office‐based visit sharply declined at the start of the COVID‐19 pandemic and did not rebound to prior levels in the study period.

LB08. Expanding the Academic Medical Center Through TelEmergency: Feasibility Study of an Acute, Layered Consultation Service

Sarah A. Sterling MD, Simon Barinas MEd, Darryl Jefferson MBA, Richard L. Summers MD

University of Mississippi Medical Center

Background: Patients in rural areas have limited access to high‐quality emergency medical care. Stroke neurologists, psychiatrists, and medical toxicologists are limited nationally, leaving rural areas disproportionately affected by shortages. TelEmergency (TE) was established to improve access to quality emergency care for patients in rural areas. Through TE, rural spoke hospitals are connected to an Emergency Medicine (EM) board certified physician at the hub site, who directs care for patients in rural Emergency Departments (EDs). Expanding TE services has the potential to improve access to specialty care, limit unnecessary transfers, and utilize limited resources in a practical manner. Our objective is to improve access to specialist care in rural areas and limit unnecessary transfers through layering an acute consultation services into our mature TE system. Initial acute services will focus on tele‐stroke, tele‐psychiatry, tele‐toxicology, which are especially limited resources in our state.

Methods: This feasibility study will utilize an existing, mature TE system and will layer acute consultation services off of this system. Specially trained advanced practice providers, at 19 rural and critical access hospital spoke sites, and the EM physician at the academic medical center hub will collaborate for usual emergency care of patients presenting to rural EDs. After evaluating the patient, the TE physician will determine if evaluation by a stroke neurologist, psychiatrist, or medical toxicologist is warranted. If deemed necessary, the tele‐health visit with tele‐stroke, tele‐psychiatry, and tele‐ toxicology with proceed, creating an extension of the academic medical center hub site. This model will improve access to specialty care not available in rural areas, but also will limit unnecessary burden on the small numbers of stroke, mental health, and toxicology specialists in our state.

Results: Currently, acute layering service consultations have not been initiated. To date, efforts have focused on consistent communication across numerous departments, providers, and hospital systems to ensure adequate understanding and training for the new consultation processes. Given the time sensitivity of patients presenting with stroke‐like symptoms, determining a thorough and reasonable stroke protocol for stroke activations at rural, spoke hospitals has been a critical component. Additionally, efforts are focused on determining an effective way to collect data across the many hospital systems and future goals of determining long term sustainability.

Discussion: By using an innovative approach which capitalizes on a mature TE system, the acute layering consultation services model has the potential to expand the services available to patients in rural area, limiting unnecessary transfers and improving care. Additionally, this system has the potential to expand coverage from stroke neurologists, psychiatrists, and toxicologists, in a practical, feasible way in state where these services and consultants are limited.

LB09. Experiences offering HIV PrEP services via a hybrid in‐person and telehealth model in Jackson, Mississippi

Kendra Johnson MPH, Erin Chase BS, Christine Khosropour PhD

University of Mississippi Medical Center, University of Washington

Background: HIV pre‐exposure prophylaxis (PrEP) is a medication to prevent HIV infection. Clinic protocols recommend quarterly clinic follow‐up visits for patients on PrEP. However, in‐person visits may be a barrier to PrEP initiation and retention. Offering patients the option of in‐person or telehealth visits may increase PrEP uptake and retention. This is particularly important in Mississippi, which has one of the highest rates of new HIV infection in the United States, but has low clinical capacity to provide PrEP.

Methods: UPrEPMS, a clinic‐based PrEP program located in Jackson, Mississippi, started offering PrEP to patients in 2018. Per UPrEPMS protocol, patients can have an initial PrEP “consultation” in‐person or via telehealth; patients interested in starting PrEP complete their baseline labs, get a prescription for PrEP, and are scheduled to see a clinical provider for a clinical evaluation. Patients can choose to have their clinical evaluation in‐person or via telehealth. In this evaluation, we describe the proportion of patients who chose to complete their consultation and clinical evaluations in‐person or via telehealth, and examined differences in process measures between these two groups using chi‐square tests. For the clinical evaluation, we describe differences in services provided to patients (PrEP education, adherence counseling, sexual risk reduction counseling, discontinuation counseling, and review of lab results) who were seen in‐person versus via telehealth.

Results: From August 2018 to September 2021 there were 199 patients who had an initial consultation to start PrEP; 40% were less than 25 years old, 73% were male sex at birth, and 73% reported being Black race. Forty‐five percent (89 of 199) of the initial consultations were completed via telehealth. There were no differences in age and race/ethnicity of individuals who had telehealth consultations vs. in‐ person consultations; however, the percent of women who opted for in‐person consultations was higher than men (68% vs. 51%, P = 0.03). Ninety‐eight percent of individuals who had a consultation were interested in starting PrEP; those who had a telehealth consultation were slightly more likely to be given a PrEP prescription (97%) compared to those who had a consultation in person (91%).

Patients with a telehealth consultation were significantly more likely to pick up their PrEP prescription compared to patients who attended an in‐person consultation (56% vs 32%; P = 0.01). There were 193 individuals who attended a clinical evaluation; 38% were telehealth visits. There were no significant differences in the types of services provided to patients when comparing those who attended an in‐ person versus telehealth clinical evaluation.

Discussion: In our PrEP clinic which offers both in‐person and telehealth visits, we observed a high uptake of telehealth visits for the initial consultation and clinical evaluation. This hybrid format of PrEP initiation and care may increase PrEP uptake and has potential to increase retention.

LB10. Use of telehealth services for prenatal care in Mississippi: Comparison of pre‐COVID‐19 pandemic and pandemic obstetric management

Jennifer C. Reneker PT,¹ Yunxi Zhang PhD,¹ Dorthy K. Young PhD,² Xiaojian Liu PhD,² Elizabeth A. Lutz MD¹

¹University of Mississippi Medical Center, ²Mississippi State Department of Health

Background: The SARS‐CoV2 (COVID‐19) pandemic resulted in major shifts in service delivery for patient care not involving COVID‐19 illness. Pre‐existing telehealth infrastructure in Mississippi allowed the state to rapidly expand the scope of telehealth programs. Little research has been done to examine the use of telehealth during the COVID‐19 pandemic and its impact on the delivery of care during and outcomes associated with pregnancy. The objectives of this study are to: 1) describe prenatal care practices during the height of the first wave of the COVID‐19 pandemic, compared to the immediate pre‐ pandemic time period; and 2) explore maternal and birth outcomes during these time periods.

Methods: This study was conducted as a retrospective historical cohort study from medical records at one Maternal Care Level IV (Regional Perinatal Health Care Center) in Mississippi and its affiliated centers. The participant cohort was inclusive of women who received prenatal care prior to a single birth delivery between May 1, 2020 and January 31, 2021. The pandemic cohort was defined through the timeframe of the included participants’ end‐term prenatal care, with reference to the beginning of the COVID‐19 pandemic. The pre‐pandemic cohort receive a majority of their prenatal care prior to the COVID‐19 pandemic.

Results: There were 1,894 women included. Among them, 620 (32.77%) completed the majority of their end‐term pregnancy in the pre‐COVID‐19 time period and 1272 (67.23%) completed the end‐term pregnancy during the pandemic. The odds for patients from the pandemic cohort of scheduling telehealth visits compared to not scheduling telehealth visits is 8.19 (95% CI 3.98, 16.86) times the odds for patients from the pre‐pandemic cohort. The pandemic exposure as well as infant gestational age, and very low birth weight (VLBW) show significant effects on the infant living status in the univariate logistic regression. However, after controlling for infant gestation age and VLBW, we did not detect a significant effect of pandemic exposure.

Discussion: This study demonstrated a very small reliance of telehealth for the medical supervision of pregnant women during the COVID‐19 pandemic. This is likely because of the essential physical examinations that occur in women who are considered high‐risk for poor maternal and birth outcomes. Additional studies on the impact of COVID‐19 infection on maternal and infant outcomes are also needed as there may be important risk factors not yet identified for poor maternal or birth outcomes.

LB11. Safety net clinics’ transition to tele‐behavioral health services during COVID‐19: The experience of the Breakwater Health Network

Rachel Cruz MPH

Great Plains Telehealth Resource & Assistance Center (gpTRAC), University of Minnesota

Background: The Breakwater Health Network is a consortium of five Community Health Centers (CHCs) with 26 sites across the upper Midwest. The majority are in mental health care health professional shortage areas (HPSAs), and many sites are in largely rural communities. Though telemedicine is beneficial in its ability to overcome provider and geographical barriers, few CHCs were offering telemedicine services before COVID‐19, and many lacked the resources to establish telemedicine programs. Furthermore, Breakwater serves a high proportion of low‐income, rural, and Latine migrant patients, and research conducted regarding utilization of telehealth among different populations during COVID‐19 suggests that these populations might have been less likely to access telemedicine during the pandemic. The Breakwater Network thus provides a case study in how telehealth access was negotiated from the patient and provider perspectives during COVID‐19 in a racially diverse, rural safety net population.

Methods: Semi‐structured interviews were conducted with behavioral health providers, administrators, and IT personnel across all Breakwater sites about their experiences transitioning behavioral health care services to telemedicine during COVID‐19. Guided by Dahlgren and Whitehead’s Social Determinants of Health Model, interviews aimed to understand how layered social, economic, geographic, political factors, among others, affected patients’ access to and clinics’ ability to offer tele‐behavioral health services. Interviewees were asked about personal and organizational priorities during the transition period, challenges and facilitators to implementing a telemedicine program, and patient receptivity to telemedicine. Data was analyzed using manual inductive and deductive coding methods to develop themes and sub‐themes. Results were analyzed at the individual and clinic levels. During the analysis, specific attention was paid to mentions of health equity or clinic efforts to maintain patient access to care, as such responses pointed to patient barriers and the clinics’ acknowledgment of and ability to address such barriers.

Results: Fourteen personnel across the clinic network were interviewed. Four of the five Breakwater clinics were offering only telemedicine visits for behavioral health services during COVID‐19. Four clinics had set up “curbside telehealth” by which patients could either park on the clinic’s property or use a designated private space within the clinic to connect to a remote provider on the clinic’s Wi‐Fi using clinic‐owned devices. Maintaining patient access to care was the most‐often reported priority, and patient willingness to use telehealth was generally high. However, one of the most often reported challenges was technology: access to necessary devices and broadband for patients and providers.

Seniors, low‐income patients, and rural residents were all reported to be generally less able to use telemedicine than other patients. Respondents had mixed views regarding Latine patients’ willingness to use telemedicine, with one respondent saying that Latine patients were more willing than others to use telemedicine, and another saying that Latine patients were less willing to use telemedicine due to privacy concerns.

Discussion: As CHCs serve largely low‐income and uninsured patients, ensuring they have access to the necessary tools to deliver quality care is paramount to reducing health disparities. While respondents reported that most of their patients were willing to try telemedicine for their behavioral/mental health care, technological barriers (such as lack of equipment or broadband) hindered accessibility of care for many, especially rural, low‐income, and senior patients. For these patients, access to “curbside telehealth” and to audio‐only telehealth (temporarily allowed during the pandemic) was highly beneficial. These findings warrant further study to determine if audio‐only telehealth is as effective for patients as the traditional audio‐visual modality. Finally, a respondent suggested that their Latine patients were concerned about privacy and the cultural stigma of accessing mental health care. More research is needed to determine how telehealth can be made more accessible to Latine patients.

LB12. Utilization of remote patient monitoring to identify weight loss and patient needs in children with poor weight gain

Courtney Sump MD, Kylee Denker BSN, Micah Dean MBA, Jennifer Ruschman ScM BS, Kenneth Tegtmeyer MD, Christine White MD, Sarah Riddle MD

Cincinnati Children’s Hospital Medical Center

Background: In January 2021, our institution implemented a novel remote patient monitoring (RPM) program for children hospitalized for failure to thrive (FTT). Eligible patients with a history of poor weight gain and followed by a participating primary care provider (PCP) are identified during admission and enrolled in RPM at discharge. Families receive a scale and mobile application that facilitates communication between the family and RPM team. Caregivers submit weights and questionnaire responses twice a week to monitor weight trends and identify barriers to following the prescribed feeding plan. RPM nurses review questionnaires, triage “alarm responses,” and notify the PCP according to a pre‐ determined escalation plan. The escalation plan defines “alarm responses” as biometrics or multiple choice answers that may benefit from clinical follow‐up. The purpose of this study was to identify the frequency of “alarm responses” and how often staff intervene with actionable recommendations.

Methods: Reviewers performed a retrospective review of patients enrolled in this RPM program from January to September 2021 by pulling questionnaire and weight reports from the remote monitoring application and a review of the electronic health record. Outcomes included total number of RPM patient weights submitted, the percentage of weights demonstrating weight loss, and “alarm responses” for the following questions: “Has your child had an increase in spit ups or vomiting today?”, “Have you been able to follow the recommended feeding plan with each feed today?”, “Are you worried you will run out of formula by the end of the week?”, “Has your child had normal wet diapers today?”, “Are you having any problems related to your child’s feeding tube today?”, and “Are you having any trouble breastfeeding?”. Additionally, responses were reviewed and assessed for the percentage of times the RPM team intervened—either by reaching out to the PCP for additional guidance or discussing a plan, such as trialing Pedialyte during acute illness, with the caregiver.

Results: Eighteen patients ranging from 10 days to 6 years old were enrolled in this program. Four patients had feeding tubes. Over this 9 month period, there were 31 occurrences of weight loss out of 313 (10%) total weights collected via RPM. Each episode of loss was forwarded to the PCP to determine whether the patient needed to be seen in clinic, if the feeding plan could be adjusted remotely, or if they could continue to monitor from home. There were a total of 64 “alarm responses” out of a total of 1,194 responses (5%). Increase in vomiting was the most frequent “alarm response” in this cohort with 29 abnormal responses, 8 (27%) of which resulted in an intervention by the RPM team. Problems related to feeding tube had 11 abnormal responses with 3 (27%) leading to intervention. Food insecurity related to formula supply was identified 9 times and in 3 (33%) of those instances RPM identified the need as urgent and connected the family with new formula supply. Wet diapers, following the recommended feeding plan, and breastfeeding troubles had 6, 5, and 4 “alarm responses” respectively, each resulting in 3 (50%, 60%, and 75%) interventions by the RPM team.

Discussion: This study examined preliminary data from a novel RPM program for patients with FTT. We measured frequency of weight loss and patient needs, including those resulting in actionable interventions. Weight loss was identified in about 10% of submitted weights, allowing PCPs to monitor trends and creating an opportunity to make changes to the feeding plan sooner than if they waited to see the patient in clinic. While “alarm responses” were rare, the RPM team was able to identify and intervene on critical alerts, such as running out of formula or signs of dehydration, which have significant implications in an infant struggling to gain weight. Many of these communications involved counseling the family on when to bring the patient into the primary care office or the emergency department. Thus, we hypothesize that these interactions may have an effect on decreasing hospital and clinic utilization, but additional patient participation and analysis are needed to make these conclusions.

LB14. Using an Evidence‐Based Treatment Protocol for Hypertension Management via Telehealth: Exploring Patterns of Protocol Deviation

Carly Brown PharmD, Julia Woods PharmD, Yunxi Zhang PhD, Donald Clark III MD

University of Mississippi Medical Center

Background: Achieving hypertension (HTN) control at a patient and population level has remained a challenge despite the availability of inexpensive medications, provider awareness and knowledge of the importance of treating HTN, and patient education factors on the benefits of treatment. Using protocols for HTN management has been encouraged by professional organizations, such as American Heart Association and Centers for Disease Control, for advantages including better blood pressure control, less variability in treatment, less cost, and accessible treatment. Recognizing the effectiveness of evidence‐based treatment protocols, existing literatures have shown negative impacts of protocol deviations on clinical outcomes. It is important to understand why protocol deviations occur in order to minimize less than optimal outcomes. The objective of this study was to examine reasons, determinants, and effects of medication intensification protocol deviations of a telehealth‐driven HTN management study.

Methods: A single‐center prospective telemedicine pilot study was conducted at the University of Mississippi Medical Center, Center for Telehealth and included 120 participants with hypertension. The feasibility of utilizing remote patient monitoring of blood pressure (BP) with protocol medication titration completed by a pharmacist via telephone was studied. Participants received equipment including an iPad and BP monitor with cuff to take daily BP measurements and complete health education sessions. At each three‐week BP review, the pharmacist determined if treatment intensification was needed based on the protocol. Participants were required to obtain at least eight BP measurements during the review cycle for intensification to occur, and an average BP of <130/80 mmHg was targeted. Data was collected using REDCap, a web‐based application for research studies. Reasons for protocol deviations were summarized. The associations between protocol deviation percentage and socio‐economic characteristics were assessed using a linear regression model. We compared mean of protocol deviation percentages by patient BP goal attainment change from baseline and evaluated the effect of protocol deviation percentages on the last BP goal attainment.

Results: Among 120 participants, 103 completed the study from August 2018 through June 2020. A total of 982 blood pressure reviews were completed by the pharmacist. Of the 982 completed BP reviews, 394 (40%) resulted in deviations from the protocol. The mean number of protocol deviations per participant was 3.31. Three main categories for protocol deviations were reported including: patient‐led factors (53.6%), justifiable clinical reasons (42.6%) and miscellaneous issues (3.8%). The most common protocol deviation rationale identified was patient reluctance to initiate or modify antihypertensive therapy. Household income and diet type of participants were independent predictors of protocol deviations. There were no significant trends regarding deviations among differing comorbid disease states or baseline number of antihypertensive medications. Patients with a greater percentage of deviations had less blood pressure control at the end of the study (OR = 0.941, p <0.001).

Discussion: In this study, protocol deviations occurred 40% of the time with patient led factors being the most common. Patients often experienced more reservation about treatment intensification than initially anticipated. Although discussions with the participants about the benefits of BP lowering and medication intensification occurred at every review, 21% of deviations were a result of participants choosing to forgo treatment intensification. Building a trusting relationship between the patient and members of the clinical team is important. In this study, data was captured during a 6‐month time frame. Due to the wide variability in the time for participants to reach a goal BP, it was determined that having flexibility in protocol length may reflect more real life chronic disease management. The study results suggest that flexibility, inter‐professional team care, and individualized patient goals are needed in addition to protocol management of hypertension.

LB15. After Hours Tele‐Skilled Nursing Facility ‐ Possible Solution to Prevent Rehospitalizations

Jeetinder Gujral MD, Joshua Case MD, Michael Perry RN, Ruchika Harsinghani MD, Ellen Pokorny DNP, Craig Hertz DO

Northwell Health

Background: Revolving door patients between Skilled Nursing Facilities (SNFs) and hospitals are a great financial burden to both hospitals and SNFs. SNFs in the post COVID era are now more likely than ever to take care of sicker patient populations. Management of changing clinical situations during off hours at skilled nursing facilities is challenging because the primary care providers are usually not available for in person bedside evaluation. This presents an obvious challenge for the nursing staff and often leads to EMS (Emergency Medical Services) being called for unscheduled emergency department (ED) transfers.

Northwell Health, with 23 hospitals, is the largest health care system in the state of New York. Northwell has a vigorous telehealth program in inpatient, and post‐acute care, and outpatient settings. The Northwell Tele‐SNF (Skilled Nursing Facility) program was implemented to provide after hour coverage service to SNF (Skilled Nursing Facility) residents. Tele‐SNF efforts are dire.

Methods: Northwell Tele‐SNF is staffed by acute care physicians, such as hospitalists and intensivists, that specialize in the care of medically complex inpatients. Tele‐SNF provides off hours’ coverage via two‐ way secure videoconference at the touch of a button upon any change in clinical condition noticed by nursing staff at our 120 bedded SNF.

Video evaluation of the patient with bedside nursing, in conjunction with review of patient electronic medical records is followed by interventions that the SNF is capable of. Interventions like medications, laboratory tests, and intravenous fluids, help us optimize management of the patient at SNF during off hours.

Workflow involves a closed loop of communication between the Tele‐SNF and the SNF providers with an email and is reviewed for meeting standards of care.

When determined to be necessary, a follow up phone call on patients retained at the SNF is made to ensure that the orders placed by Tele SNF provider were carried out successfully.

Ongoing education the nursing staff at SNF is done to ensure their comfort with technology and processes of Tele‐SNF. Recurring reminder of capabilities available at SNF are shared with Tele‐SNF providers to ensure foremost utilization.

Results: Over the 9‐month study at our 120 bed SNF, we had 131 unique patient interactions. Of the 131, the most common diagnoses were Sepsis (19), Hypertension (11), Altered Mental Status (9), and Fall (8). We have 127 complete records noting 111 (87.4%) patients required on‐site interventions (medications, Intra venous fluids or laboratory radiology testing), with 85 (66.9%) remaining at the SNF, 39 (30.7%) being transferred to an Emergency Department, and 3 (2%) refusing transfers.

Furthermore, 39 (45.8%) patients who remained had a scheduled follow‐up evaluation by the Tele‐Hospitalist team with 4 (4.7%) being transferred after repeat evaluation. Of the 39 transfers, 22 (56%) were admitted, and 17 were returned to the SNF. Of the patients returned to the SNF, most required either advanced imaging (i.e., CT or ultrasound) or more rapid laboratory evaluation (i.e., cardiac biomarkers) at the emergency department prior to transfer back to the SNF.

Discussion: Over the 9‐month study at our 120 bed SNF, we had 131 unique patient interactions. Of the 131, the most common diagnoses were Sepsis (19), Hypertension (11), Altered Mental Status (9), and Fall (8). We have 127 complete records noting 111 (87.4%) patients required on‐site interventions (medications, Intra venous fluids or laboratory radiology testing), with 85 (66.9%) remaining at the SNF, 39 (30.7%) being transferred to an Emergency Department, and 3 (2%) refusing transfers.

Furthermore, 39 (45.8%) patients who remained had a scheduled follow‐up evaluation by the Tele‐ Hospitalist team with 4 (4.7%) being transferred after repeat evaluation. Of the 39 transfers, 22 (56%) were admitted, and 17 were returned to the SNF. Of the patients returned to the SNF, most required either advanced imaging (i.e., CT or ultrasound) or more rapid laboratory evaluation (i.e., cardiac biomarkers) at the emergency department prior to transfer back to the SNF.

LB16. Estimated Population Health and Economic Impact of Statewide Implementation of the University of Mississippi Medical Center and TeleHealth Center Hypertension Remote Patient Management Program

William B. Hillegass MD, Yunxi Zhang PhD, Loretta Cain‐Shields PhD, Alan Barefield PhD, William G. Davis PhD, Donald Clark MD

University of Mississippi Medical Center, Mississippi State University

Background: Optimal blood pressure control is considered the most effective approach to reducing adverse cardiovascular disease and events. Suboptimal blood pressure is commonplace and results in excess cardiovascular mortality, nonfatal myocardial infarction (MI), nonfatal stroke (CVA), incident heart failure (HF), and advanced kidney disease (ESKD). The UMMC Telehealth Center’s hypertension remote patient management program enrolled 100 patients with SBP ≥140mmHg and/or DBP ≥90 mmHg on two clinic visits while excluding those with Stage 4 or 5 kidney disease, CHF with LVEF <50%, CV event within the prior 3 months, and/or pregnancy. Over one year, the average observed decrease in SBP of ‐14.1 (95% CI: ‐16.8, ‐11.4) mmHg and DBP of ‐7.9 (95% CI: ‐9.5, ‐6.4) mmHg were significantly greater than a well‐matched control group.

Methods: Mississippi’s almost 3 million population was simulated at 1:1 scale. Each simulated individual resides geographically within one of Mississippi’s 748 census tracts, with demographically correct distribution by age, gender, race, household income, employment, educational attainment, and payor for each census tract. Each individual is assigned three simulated blood pressure (BP) recordings based on their individual history of diagnosed hypertension, medications, BMI, diabetes status, smoking, HgbA1C, estimated GFR, lipid profile, and prevalent CAD, prior MI, CVA, HF, end‐stage kidney disease (ESKD). Each individual’s 10 year risk for all‐cause mortality, nonfatal MI, nonfatal CVA, incident HF, and incident renal replacement therapy are modeled as a function of BP and their individual characteristics. The 10‐year influence of improved BP control with the RPM on event risk was estimated from meta‐analyses, given baseline BP and subject characteristics. Incremental cost of events were estimated from the Medical Expenditure Panel Survey. Incremental cost of the RPM was estimated from the demonstration project. Monetary values are expressed as net present value (NPV) in 2018 US dollars.

Results: 16 ± 2% of adult Mississippians (364,000 [95% Cl: 329,000 to 399,000]) are estimated eligible for the HTN RPM. Assuming 28.4% non‐participation (observed rate = 14.2%), random draws from the RPM distribution of reduced SBP of ‐14.1 +/‐ 1.4 mmHg and DBP of ‐7.9 +/‐ 0.8 mmHg adjusted for individual characteristics is applied to the eligible willing participants. Over 10 years, the simulation forecasts 7,200 [95%CI: 6,100, 8,300] fewer nonfatal CVAs, 56,300 [95%CI: 46,200, 66,400] fewer deaths, 13,000 [95%CI: 10,400, 15,700] fewer nonfatal MIs, 7,800 [95%CI: 6,200, 9,400] less HF diagnoses, and 4,900 [95%CI: 3,900, 6,000] fewer ESKD cases. Reduction or delay in events yields an estimated incremental direct medical cost savings of $2,860,000,000 NPV 2018 USD with a 3% annual discount rate. Assuming 71.6% of the eligible participate, year one program costs are estimated at $156,000,000. With 25% annual rate of re‐referral for subsequent BP control deterioration,10‐year program incremental NPV costs are estimated at $455,000,000. With estimated reduction in future direct medical costs exceeding incremental RPM cost, net direct medical cost savings are forecast at $2,400,000,000 NPV realized over 10 years for Mississippi.

Discussion: Improved HTN control remains a promising and well‐documented direct approach to improving cardiovascular outcomes. Consistent with randomized trials, the UMMC HTN RPM had high adherence and significantly improved blood pressure control. A detailed simulation of the demographics and cardiovascular health of Mississippians estimates that approximately 16% of adult Mississippians would meet eligibility criteria for the HTN RPM intervention. Forecasting the incremental impact of the RPM on 10 year cardiovascular events suggests significant reductions in death, MI, CVA, HF, and ESKD would be achieved. Prevented and delayed events would likely yield direct medical cost savings well exceeding (five‐fold) incremental program costs. A statewide HTN RPM might initially target specific patient characteristics such as baseline BP that identify Mississippians with higher than average predicted improvement in health outcomes and cost savings. This would also reduce observed health disparities.

LB17. Leveraging Project ECHO to provide telehealth education during the COVID‐19 pandemic

Shannon Limjuco MPH, Trisha Calabrese MPH, Lauren Geary MPH, Oyinkansola Oloniniyi MPH

American Academy of Pediatrics

Background: The COVID‐19 pandemic resulted in tremendous changes in health care delivery, including an increase in pediatric providers utilizing telehealth. In partnership with the Health Resources and Services Administration (HRSA), the AAP leveraged Project ECHO to support telehealth education for pediatric providers during the COVID‐19 pandemic. Project ECHO is a tele‐mentoring program designed to create communities of learners by bringing together health care providers and experts in topical areas using didactic and case‐based presentations, fostering an “all teach, all learn” approach. Two AAP ECHO programs focused on behavioral and mental telehealth care and adolescent telehealth care, respectively; each engaging 25‐50 pediatric providers in learning and quality improvement activities. The AAP also supported 12 AAP Chapters throughout the United States to develop and implement their own telehealth ECHO programs, ultimately engaging 250 participants in 72 ECHO sessions across the Chapters.

Methods: The AAP Adolescent Telehealth Care ECHO engaged 13 pediatric practices across 8 states in the 6‐session program designed to build capacity on use of telehealth to improve adolescent health promotion within the medical home. Practice also engaged in quality improvement (QI) aimed at increasing compliance with preventive care guidelines, including depression screening, discussing confidentiality, and facilitating time alone with the clinician. The Behavioral and Mental Health Telehealth Care ECHO engaged 3 practices in the 6‐session program to build capacity on use of telehealth to improve behavioral and mental health promotion within the medical home. Practices engaged in QI aimed at increasing compliance with preventive care guidelines, including performing and following up on psychosocial/behavioral assessments and eliciting/addressing patient/family concerns. Twelve AAP Chapters developed a 6‐session Telehealth ECHO program for their members. The AAP provided training, technical assistance, and support for the chapters to each develop a scalable and adaptable telehealth ECHO program. Evaluation methodology included collecting data on participation, post session surveys, retrospective post‐program survey, and focus groups.

Results: Results demonstrated that practices participating in the Adolescent Telehealth ECHO increased their discussion of confidential care and delivery of time alone with adolescent patients. Many practices incorporated prompts to document time alone and confidential care. In addition, practices found telehealth an effective tool for psychosocial follow up visits. Practices participating in the Behavioral and Mental Health Care ECHO reported improvements in their follow up for psychosocial/behavioral health assessments. Participants also reported an increased understanding of how to utilize assessment tools via telehealth. AAP Chapters reported high satisfaction from participants in their respective ECHO programs. Chapter ECHO participants reported that the ECHO format of utilizing case discussions encouraged creative approaches to telehealth, acknowledging the limitations, yet recognizing the role of telehealth care today and in the future. In addition, participants reported changes to their practice workflow to incorporate telehealth in their daily schedule.

Discussion: This project demonstrates that the ECHO model is an effective and efficient way to disseminate information and provide support for healthcare professionals to provide telehealth during a pandemic and beyond. Using the ECHO model offers advantages over traditional training methods and provides a responsive approach to the changing pediatric practice delivery. In particular, participants reported that the utilization of ECHO case‐based learning, was valuable to their shift to providing effective telehealth care.

LB18. Training Providers and Educators in Telehealth Implementation, and Evaluation

Tina Gustin DNP, Carolyn Rutledge PhD

Old Dominion University

Background: Telehealth was rapidly implemented to address barriers during the COVID pandemic. Waivers allowed for telehealth practice to be conducted without the customary guidelines, regulations, and training. This eliminated barriers to telehealth implementation and enabled many patients and providers to develop an appreciation for telehealth delivery. However, the rapid expansion of telehealth due to the pandemic left many providers conducting telehealth visits without the necessary skillset to implement and maintain a safe, effective, and viable practice. It has become evident that provider training in telehealth is a must for telehealth to continue. Proper training must be nested in a framework and focus on competency development. As telehealth has expanded, some states now require proof of education in telehealth for reimbursement. The American Association of Colleges of Nurses (AACN) and the AAMC now require that faculty include telehealth content within the curriculum. This can be problematic in that most faculty are not experienced in telehealth education. This presentation will discuss the development of and outcomes from two 20‐hour asynchronous telehealth certification courses. The courses are framed on the 4 Ps of Telehealth Framework (Planning, Preparing, Providing, and Performance Evaluation) developed by the authors. The courses are designed for faculty, students, and providers.

Methods: During a 20‐hour asynchronous online program, students, faculty, and providers from different professions are guided through modules where they learn about key telehealth content such as: planning and preparing for a telehealth encounter, telehealth delivery modalities/equipment, legal and regulatory issues, reimbursement, conducting virtual physical assessments, telehealth etiquette, and methods for evaluating telehealth programs using the SPROUT and NQF telehealth frameworks. Throughout the program, students view a series of videos developed by the authors on conducting physical assessments without peripherals, providing telehealth education to patients using the four learning styles, preparing the patient for a telehealth visit, and utilizing appropriate telehealth etiquette. Participants learn to evaluate the learning as well as the telehealth delivery based on the telehealth metrics. At the completion of the program students have a scheduled virtual encounter with a Standardized Patient (SP) where they practice their telehealth skills followed by structured feedback. The SP cases have been designed to match the learner’s profession (i.e., ankle pain for physical therapy, congestive heart failure medication management for pharmacy, bowel pain for primary care providers). Throughout the course validated tools are used to measure participant knowledge, etiquette, and telehealth assessment skills. At the completion of the program, students receive a certification in telehealth.

Results: Over 1500 participants from throughout the United States have completed the program. Five universities are utilizing the program as part of their nurse practitioner curriculum. Over 20 universities have sent students and faculty through the program. Many of the participants have since developed either telehealth educational programs for students or telehealth delivery program. There has been a significant increase in the participants’ confidence in providing telehealth. Students in the Doctor of Nursing Practice (DNP) program that participated in the program have developed evidence‐ based research projects that have shown significant improvements in patient access, outcomes, and satisfaction. One specific area of DNP research addressed virtual palliative care for children. The self‐ efficacy of caregivers increased, and hospitalizations and cost decreased in the pediatric palliative care project. Specific project and outcomes will be addressed during the presentation.

Discussion: The incorporation of the Four Ps of Telehealth framework has successfully guided the program design as well as the participant learning. Participants at all levels have incorporated telehealth into practice as a result of course completion. The authors have validated the instruments developed for this program and have made them available for other telehealth program use. Specific tools include: Confidence in Delivering Telehealth, the Teaching Interpersonal Skills in Telehealth (TIPS) scale, and Conducting a Physical Assessment through Telehealth. The videos have been made publicly available. The assessment and etiquette videos are open access on the AAMC iCollaborative website. This program has served as a model for delivering a comprehensive program for providers and educators across the nation.

LB19. Acceptability of the Tele‐Lactation Consultation Service: A Randomized Feasibility Study

Cari A. Bogulski PhD,¹ Nalin Payakachat PhD,¹ Sarah Rhoads PhD,² Rebecca Jones, MPH,² Hannah McCoy, BA,¹ Leah Dawson PhD,¹ Hari Eswaran PhD¹

¹University of Arkansas for Medical Sciences, ²University of Tennessee Health Science Center

Background: The American Academy of Pediatrics and the American College of Obstetricians and Gynecologists recommend that all infants be exclusively breastfed for the first six months of life. There are many health benefits to both the mother and child when the child is breastfed.

Methods: Research Aim: To assess and compare feasibility and patient acceptability of tele‐lactation services implemented via telephone or audio‐visual, using quantitative and qualitative methods.

Methods: A prospective, randomized design with two arms (telephone vs. audio‐visual) was used in this pilot study with a convenience sample of postpartum women recruited from two medical centers in Little Rock, Arkansas. Participants completed a pre‐ and post‐study survey and interview assessing demographic characteristics, breastfeeding knowledge, perceived social support, pregnancy‐related care and patient acceptability of tele‐lactation services.

Results: Participants were randomized into telephone‐only and audio‐visual intervention groups. No significant difference in demographic characteristics across groups was found. At 3‐months after discharge, both groups reported continued breastfeeding (telephone‐only: 81% vs. audio‐visual: 90%) with no significant difference between the two groups (χ2 = 0.14, p = 0.706). ANCOVA analyses for group differences on breastfeeding knowledge controlling for breastfeeding knowledge at baseline did not reveal a significant difference (t = ‐0.59, p = 0.439). No significant difference on perceived social support (t = 0.17, p = 0.864) was observed. Overall, participants reported positive experiences with tele‐lactation, emphasizing the convenience, accessibility, education, and support provided.

Discussion: Tele‐lactation services offer a promising solution to encourage and sustain breastfeeding behavior in mothers, particularly in rural areas.