Innovative Use of a Mobile Web Application to Remotely Monitor Nonhospitalized Patients with COVID-19
Introduction
COVID-19 presents with a range of clinical manifestations1,2 and can have an unpredictable course, with some patients experiencing rapid worsening after many days of stability.3 Most patients with COVID-19 do not need to be hospitalized,2 but because of the potential for deterioration, recently diagnosed outpatients need monitoring, which may strain already overwhelmed outpatient practices.4 Typically, a primary care practice might have a few patients to check in with each day, perhaps following a hospital stay,5 or when managing an acute exacerbation of a chronic condition.6
With the onset of COVID-19, however, many practices suddenly had to outreach to many patients daily to monitor symptoms and provide ongoing medical advice.7,8 Tracking and calling these patients takes time, logistical expertise, and clinical resources, all of which may be in short supply during a pandemic.
Some outpatient clinicians already use mobile applications and automated text messaging systems to communicate with patients and to support them in self-managing a range of chronic conditions.9–13 Examples in the primary care setting include tobacco cessation,10 depression,11 and blood pressure and glucose monitoring.12,13 Mobile applications and automated text messaging are used less often for individualized outpatient management of acute conditions14,15 but may be well-suited to address the needs of overwhelmed health care systems during a pandemic.16–20
With this in mind, a mobile web application already used by primary care practices to help manage chronic conditions was adapted to support the care of patients with suspected or confirmed COVID-19. The aims of this study were to (1) describe the implementation of a mobile web application to monitor COVID-19 signs and symptoms among nonhospitalized patients at primary care practices and (2) assess the feasibility and acceptability of the application from the perspective of both the patients and the practices. We hypothesized that implementation of the mobile web application would be feasible, that use would be acceptable to clinicians and patients, and that the application would be viewed as a valuable addition to practices’ COVID-19 outreach strategies.
Materials and Methods
Study Design and Setting
This study is a retrospective analysis of (1) mobile web application data from March 21, 2020, through December 31, 2020, and (2) cross-sectional surveys administered to participating patients and primary care practice staff in June 2020. Nine practices at 29 sites located in Massachusetts and Rhode Island implemented the application. Participating practices included community health centers, multispecialty private practices, a free clinic, and a solo practice.
Description of Mobile Web Application
Healthcentric Advisors, a nonprofit health care quality improvement organization, initially developed the application in early 2020 to support care of patients with diabetes, hypertension, and heart failure. With the onset of COVID-19, the application was adapted to facilitate remote outpatient monitoring of signs and symptoms of nonhospitalized patients with suspected or confirmed COVID-19. The purpose of the application was to allow staff at the practices to remotely track patients with COVID-19-related symptoms and to prioritize daily outreach to their highest risk patients because the number of patients needing monitoring exceeded the capacity of the clinical staff. In addition, information gathered from the application was used to guide communication by clinical staff to patients’ primary care clinicians.
Patients using the application received an automated daily text message at 9am, with a reminder at noon if no response had been recorded. The text message connected patients to a web-based applet, which prompted them to report on five domains: (1) their breathing (normal, better, not improved, worse), (2) recent temperature (up to 99.4 F, 99.5 to 100.3 F, 100.4 F and above, not taken), (3) use of medications to manage symptoms (yes, no), (4) cough (no cough, better, not improved, worse), and (5) oxygen saturation (only available for a subset).
If patients’ answers were not clinically concerning, an auto-response instructed them to contact the practice or call 911 if their condition changed. If any of the responses met a predetermined threshold for telephone outreach, an auto-response informed the patients that their care team had been alerted. The application was available in English, Spanish, and Portuguese.
Figure 1a shows a screen shot of the application from the patient perspective. Figure 1b shows a screenshot of the practice dashboard, which had aggregate data for all patients using the application in a particular practice at a given time. The mobile web application was customized with the practice name and contact information, to make the interface recognizable to patients.
Implementation of Mobile Web Application by Primary Care Practices
Most practices enrolled patients during an existing telehealth visit with a clinician. Some practices only enrolled patients with a confirmed COVID-19 diagnosis; others also included those with a COVID-19 exposure or those waiting for a COVID-19 test result. Practices could opt to categorize enrolled patients by risk level; risk levels were determined by the practice.
If patients reported their breathing was “worse,” the program flagged the response for telephone outreach by generating an email alert to the practice. An option for patients to request that practice staff contact them was later added to the application, and this also generated an email alert. Each practice designated a staff member to receive the alerts. Many practices also had staff create an encounter in the electronic health record to send to patients’ primary care clinicians notifying them of the alert and, depending on the clinical scenario, requesting an in-person or telehealth visit.
Practice staff could also monitor all patients enrolled in the program via a dashboard in a web-based portal (Fig. 1b). The dashboard graphically displayed each patient’s responses over time. Patients’ signs and symptoms and the staff response could be downloaded into a PDF for inclusion in the electronic health record. Practices varied in whom they designated to receive the alerts, to monitor the dashboard, and to call patients; most chose nurses, care managers, pharmacists, or advance practice providers. Healthcentric Advisors staff trained practices on how to enroll patients, how to navigate the dashboard, and best practices for responding to alerts.
Practices paid a monthly subscription to use the mobile web application; the amount varied based on the number of patients actively monitored each month. For example, a practice monitoring up to 150 patients in a month would be charged $200 for that month. Of note, during the study period, onboarding costs and charges for the first several months were waived.
Survey Design and Administration
Two months after implementation, Healthcentric Advisors staff administered a survey to participating patients and practice staff to assess their initial experiences, with the goal of eliciting feedback to guide adjustments to the program. The survey was administered in June 2020 via SurveyMonkey (San Mateo, CA). Patients were asked the extent to which they agreed or disagreed with the following statements: “Responding to texts about my COVID-19 symptoms was easy,” “Participating in the COVID-19 Text Monitoring allowed me to report and manage my symptoms outside a doctor’s office visit,” “I felt better connected to my provider while using the COVID-19 Text Monitoring,” “I feel supported knowing my provider is checking in on my symptoms during the pandemic,” and “I would be willing to participate in other texting campaigns with my provider and care team in the future.”
Practice staff were asked the degree to which they agreed or disagreed with the following statements: “Utilizing the COVID-19 Remote Patient Monitoring tool has reduced the burden of outreach by practice staff,” “I feel comfortable making treatment changes based on data received through the portal,” “I would recommend participation in the CMAssist COVID-19 Campaign to other providers,” and “The Insights Dashboard has been a valuable tool to manage our entire panel at a glance.” A free text field captured whether having to log into the system portal was “worth the effort of leaving your EHR.”
Statistical Analysis
We used descriptive statistics to characterize the enrolled patients, including age, preferred language, assigned risk level (if present), reason for enrollment, whether a COVID-19 test had been performed, and the presence of a positive test. Gender was not collected. Practices did not track the number of patients who declined enrollment.
The primary outcome was the proportion of texts during the study period that resulted in a response from participants. Secondary feasibility and acceptability outcomes from the patient perspective included the average proportion of patients who responded to texts each day, the proportion of patient survey respondents who “agreed” or “strongly agreed” that responding to texts was easy, that participating allowed them to manage symptoms, that they felt connected and supported, and that they were willing to participate in other health care texting programs.
From the practice perspective, secondary feasibility and acceptability outcomes included the median number of alerts per patient, the proportion of patients triggering an alert, the proportion of all responses that generated an alert, the proportion of staff survey respondents who “agreed” or “strongly agreed” that the application reduced outreach burden, that they felt comfortable making treatment decisions based on information received, that they recommended use of the application to other practices, and that the dashboard was a valuable tool for panel management. All outcomes were identified a priori.
Stata (College Station, TX) was used for data analysis. The Lifespan Institutional Review Board reviewed the study protocol and determined that it was exempt.
Results
Sample Characteristics, Signs, and Symptoms
Practices enrolled their first patient on March 21, 2020. By the end of the study period, 5,532 patients had used the application across 29 different practice sites, with a total of 26,466 responses. The median number of patients enrolled in a practice at any one time was 3 (interquartile range [IQR] 1–8); the maximum number followed by any practice at any one time was 96. The mean patient age was 49.1 years old (Table 1); 28% of participants were 60 or older (Table 1).
CHARACTERISTICS | n = 5,532 |
---|---|
Age in years, mean (SD) | 49.1 (16.2) |
Preferred language, n (%) | |
English | 5,364 (97.0%) |
Portuguese | 4 (0.1%) |
Spanish | 164 (3.0%) |
Reason for enrollment, n (%) | |
Tested positive | 621 (11.2%) |
Awaiting testing | 2,123 (38.4%) |
Awaiting test results | 2,499 (45.2%) |
Exposure or presumed positive (without testing) | 289 (5.2%) |
Assigned risk levela, n (%) | |
Low | 1,581 (28.6%) |
Medium | 111 (2.0%) |
High | 1,602 (29.0%) |
COVID-19 test ever performed, n (%) | 1,625 (29.4%) |
Any positive, n (%) | 881 (15.9%) |
Signs and symptoms during monitoringb, n (%) | |
Worsening breathing | 305 (5.5%) |
Temperature of 100.4 or above | 173 (3.1%) |
Worsening cough | 460 (8.3%) |
Oxygen saturationc <95% | 77 (1.4%) |
Use of medications to manage symptoms | 3,415 (61.7%) |
Among the patient participants, 6% reported at any point during their monitoring period that they had worsening breathing, and 8% reported worsening cough (Table 1). Among all of the responses received during the monitoring period, about 2% included a report of worsening cough and 1% included a report of worsening breathing, a temperature of 100.4, or an oxygen saturation below 95%.
Use of Mobile Web Application and Alerts Generated
Patients were monitored for a median of 7 days (IQR 5–14), with a range of 0 to 313 days. Overall, 78% of the texts resulted in a response from participants. About three-quarters of patients responded to the text each day (median daily proportion 77%).
Seven percent of patients had an alert sent to the practice based on their responses. Each patient generated a median of 0 alerts (IQR 0–0) while enrolled, with a maximum of 4 alerts per patient noted in the sample. Fewer than 2% of all the responses generated an automated alert to the practice during the study period. Practices received a median of 0.32 alerts (IQR 0–0.60) per week, with a median of 0.04 alerts (IQR 0–0.10) per clinician per week.
Survey Results
Among the 1,887 surveyed patients, 408 responded (response rate 22%). Almost all patients “agreed” or “strongly agreed” that responding to texts was easy (95%) and that they felt supported knowing the practice is checking in (90%) (Fig. 2a). Among practice staff, 18 responded to the survey from 4 practices (response rate 44% of practices). Most staff “agreed” or “strongly agreed” that the program reduced the burden of outreach for the practice (94%); 100% would recommend other practices use the application (Fig. 2b). Seventeen of the 18 responded “yes” that it was worth logging into a system outside of their electronic health record (the remaining respondent left the field blank).
Discussion
We found that use of a COVID-19 symptom tracking application was feasible and acceptable to patients and primary care practice staff. Use of the application reduced outreach burden for primary care practices and had high engagement from patients. Three-quarters of the daily texts resulted in a response during the study period. Patients reported that the mobile web application was easy to use, helped them manage their symptoms, and made them feel more connected and supported by the practice. Practice staff reported that the application allowed them to safely prioritize clinical interventions for patients whose condition was worsening; 100% of practice survey respondents would recommend the application to other physician practices.
The COVID-19 pandemic has sparked a number of technological innovations, including mobile web applications like ours, as well as COVID-19-related applications designed for use by individuals. These applications have varying objectives: providing health information, assisting in contact tracing, facilitating symptom tracking for personal use, providing support for COVID-19-related mental health challenges, and monitoring health care worker outbreaks.18,21–28 Other applications predict which individuals will develop COVID-19, based on reported symptoms, particularly loss of taste or smell, or even by recording users’ coughs.29,30 Another category of applications is also aimed at individuals, but the purpose is to provide data at the regional level for hot-spotting and predicting outbreaks,31,32 as has been done with other infectious diseases such as influenza.33
Despite the explosion in COVID-19-related applications, we did not find another example of an exclusively outpatient practice using an application for COVID-19 tracking and managing panels of COVID-19 patients, as in this study. The most similar program was administered through the US Department of Veterans Affairs (VA).34 The VA adapted an existing self-management text messaging program and used it to deliver COVID-19 education and to check in with patients every other day. The application messaged participants to ask if they were feeling well; if a patient responded “no,” they were prompted for additional information, and, if concerning, they received an automated message instructing them to call their clinician.
This program differed from ours in that (1) all veterans were invited to enroll instead of targeting specific patients and (2) nobody was monitoring responses. Veterans who used the program felt that the advice made them feel more connected to the VA. The authors noted that the program likely led to cost savings because it reduced messages from patients and calls by clinical staff.34
While our application was used by primary care practices, large hospital systems, and even nations, harnessed similar technology to manage increasing volumes of patients during the COVID-19 pandemic. Iceland and Ireland initially supported positive cases or close contacts by having clinicians call patients, but both countries quickly became overwhelmed, and each then implemented mobile applications that instructed patients with concerning symptoms to call their own physicians for further management.18,19
South Korea and China monitored state-quarantined COVID-19 patients using multifaceted telemedicine interventions that relied on a centralized group of clinicians to answer questions and review vital signs transmitted from wearable technology, among other innovations.16,17 Hospitals in Europe have used mobile applications to follow COVID-19 patients after hospital discharge.35,36
Limitations
The strengths of this study include a large sample size and implementation across a range of practices. In addition, we use data from the application itself—so we did not have to rely on self-report regarding use—as well as surveys to capture both the patient and practice perspective. Several limitations should also be noted. First, the application was implemented in one region of the country and may not be generalizable to regions with different practice patterns. In addition, although the application was available in Spanish and Portuguese, the patients primarily used the English-language version, which may limit generalizability to more diverse patient populations.
We recognize that use of applications like this one may worsen the digital divide if marginalized groups do not have equitable access to both technology and broadband services.37–39 In a South Korean study of an application that helped people decide whether they need COVID-19 testing, researchers found that use was high among older people and groups with low digital literacy,40 but more data are needed in diverse US populations.
Second, patients who agreed to use the application are likely different in unmeasured ways from those who declined, and practices that implemented the application are also likely different from practices that did not. For example, participating practices may operate other remote monitoring programs and they may have more support staff.
Third, we do not have clinical data and thus cannot assess the impact on important outcomes such as hospitalization. Fourth, survey response rates for practices and patients were relatively low, and it is likely that respondents differed from nonrespondents. Of note, several practices stopped using the program when their numbers of patients with COVID-19 dropped, reporting that the lower patient numbers allowed for individual tracking by nurses; this indicates that there is a likely a to-be-determined prevalence of disease needed to make a program like this feel worthwhile to practices.
Conclusion
In conclusion, we found that primary care practices successfully implemented a mobile web application to track COVID-19-related signs and symptoms among their nonhospitalized patients, allowing the practice staff to prioritize outreach to the sickest patients. Use of the COVID-19 symptom tracking application was feasible and acceptable to patients and to practice staff, and it eased burdens on practice staff, who were feeling overwhelmed by competing priorities during the pandemic.
Use of the application likely saved the practice time, but formal time and cost analyses are needed. We also recommend trials comparing use of the application to use of daily outreach phone calls, with an emphasis on patient-centered and other clinical outcomes. Even when the pandemic wanes, our findings apply to other prevalent, acute illnesses that need close outpatient monitoring and to outpatient practices that want to expand remote monitoring of chronic diseases but may not have felt they had the staff do so.
Authorship Confirmation Statement
All authors confirm that they have made substantial contributions to the work, have participated in drafting or revising the article, have given final approval for submission, and have agreed to be accountable for the content.
Acknowledgments
We thank the patients and practices for their enthusiastic participation, with particular gratitude to Coastal Medical. We also thank Bryan Los for his assistance in developing the program’s texting capabilities and adapting the mobile web application for COVID-19 monitoring. Finally, we thank Susan Ramsey, Christopher Campanile, and Melissa Miranda for their insightful review of the article.
Disclosure Statement
Healthcentric Advisors, the employer of J.H., B.J., L.F.C., E.L.C., and B.M., developed the mobile application as a community service during the COVID-19 pandemic, with the assistance of a grant from anonymous donor. R.L.G. serves as a consultant to Healthcentric Advisors. The authors have no other disclosures.
Funding Information
A one-time grant from an anonymous donor funded the onboarding and initial costs for the participating practices; there were no other external funding sources.
References
- 1. Immune response, inflammation, and the clinical spectrum of COVID-19. Front Immunol 2020;11:1441. Crossref, Medline, Google Scholar .
- 2. Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease 2019 (COVID-19): A review. JAMA 2020;324:782–793. Crossref, Medline, Google Scholar .
- 3. Clinical presentation and management of COVID-19. Med J Aust 2020;213:134–139. Crossref, Medline, Google Scholar .
- 4. https://static1.squarespace.com/static/5d7ff8184cf0e01e4566cb02/t/5ebe8e3aa633d80ce5e7c107/1589546555002/C19+Series +9+National+Executive+Summary+with+comments.pdf (last accessed
August 21, 2021 ). Google Scholar , 2020. Primary Care Collaborative and Larry A Green Center. Available at - 5. Evaluation of a primary care-based post-discharge phone call program: Keeping the primary care practice at the center of post-hospitalization care transition. J Gen Intern Med 2014;29:1513–1518. Crossref, Medline, Google Scholar .
- 6. Action plans with brief patient education for exacerbations in chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2016;12:Cd005074. Medline, Google Scholar .
- 7. Retooling primary care in the COVID-19 era. Mayo Clin Proc 2020;95:1831–1834. Crossref, Medline, Google Scholar .
- 8. Covid-19: A remote assessment in primary care. Br Med J 2020;368:m1182. Crossref, Medline, Google Scholar .
- 9. Effectiveness of an app and provider counseling for obesity treatment in primary care. Am J Prev Med 2018;55:777–786. Crossref, Medline, Google Scholar
- 10. Mobile phone text messaging and app-based interventions for smoking cessation. Cochrane Database Syst Rev 2019;10:Cd006611. Medline, Google Scholar .
- 11. Coached mobile app platform for the treatment of depression and anxiety among primary care patients: A randomized clinical trial. JAMA Psychiatry 2020;77:906–914. Crossref, Medline, Google Scholar
- 12. New approaches in hypertension management: A review of current and developing technologies and their potential impact on hypertension care. Curr Hypertens Rep 2019;21:44. Crossref, Medline, Google Scholar .
- 13. Effectiveness of mobile app-assisted self-care interventions for improving patient outcomes in type 2 diabetes and/or hypertension: Systematic review and meta-analysis of randomized controlled trials. JMIR Mhealth Uhealth 2020;8:e15779. Crossref, Medline, Google Scholar .
- 14. Effects of a home care mobile app on the outcomes of discharged patients with a stoma: A randomised controlled trial. J Clin Nurs 2018;27:3592–3602. Crossref, Medline, Google Scholar
- 15. Implementation of a mobile app for trauma education: Results from a multicenter study. Trauma Surg Acute Care Open 2020;5:e000452. Crossref, Medline, Google Scholar
- 16. Information technology-based management of clinically healthy COVID-19 patients: Lessons from a living and treatment support center operated by Seoul National University Hospital. J Med Internet Res 2020;22:e19938. Crossref, Medline, Google Scholar
- 17. Monitoring and management of home-quarantined patients with COVID-19 using a WeChat-based telemedicine system: Retrospective cohort study. J Med Internet Res 2020;22:e19514. Crossref, Medline, Google Scholar
- 18. Beating the odds with systematic individualized care: Nationwide prospective follow-up of all patients with COVID-19 in Iceland. J Intern Med 2021;2:255–258. Crossref, Google Scholar
- 19. Measuring the effectiveness of an automated text messaging active surveillance system for COVID-19 in the south of Ireland, March to April 2020. Euro Surveill 2020;25:2000972. Crossref, Google Scholar .
- 20. COVIDApp as an innovative strategy for the management and follow-up of COVID-19 cases in long-term care facilities in Catalonia: Implementation study. JMIR Public Health Surveill 2020;6:e21163. Crossref, Medline, Google Scholar
- 21. Changes in stress, anxiety, and depression levels of subscribers to a daily supportive text message program (Text4Hope) during the COVID-19 pandemic: Cross-sectional survey study. JMIR Ment Health 2020;7:e22423. Crossref, Medline, Google Scholar
- 22. A systematic review of smartphone applications available for corona virus disease 2019 (COVID19) and the assessment of their quality using the Mobile Application Rating Scale (MARS). J Med Syst 2020;44:164. Crossref, Medline, Google Scholar
- 23. Digital response during the COVID-19 pandemic in Saudi Arabia. J Med Internet Res 2020;22:e19338. Crossref, Medline, Google Scholar .
- 24. Mobile health apps on COVID-19 launched in the early days of the pandemic: Content analysis and review. JMIR Mhealth Uhealth 2020;8:e19796. Crossref, Medline, Google Scholar
- 25. Evaluation of the design and implementation of a peer-to-peer COVID-19 contact tracing mobile app (COCOA) in Japan. JMIR Mhealth Uhealth 2020;8:e22098. Crossref, Medline, Google Scholar
- 26. A review and content analysis of national apps for COVID-19 management using Mobile Application Rating Scale (MARS). Inform Health Soc Care 2021;46:42–55. Crossref, Medline, Google Scholar .
- 27. Hospital Epidemics Tracker (HEpiTracker): Description and pilot study of a mobile app to track COVID-19 in hospital workers. JMIR Public Health Surveill 2020;6:e21653. Crossref, Medline, Google Scholar
- 28. Digital tools to ameliorate psychological symptoms associated with COVID-19: Scoping review. J Med Internet Res 2020;22:e19706. Crossref, Medline, Google Scholar .
- 29. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med 2020;26:1037–1040. Crossref, Medline, Google Scholar
- 30. A COVID-19 multipurpose platform. Digit Biomark 2020;4:89–98. Crossref, Medline, Google Scholar .
- 31. Population-scale longitudinal mapping of COVID-19 symptoms, behaviour and testing. Nat Hum Behav 2020;4:972–982. Crossref, Medline, Google Scholar
- 32. Detecting COVID-19 infection hotspots in England using large-scale self-reported data from a mobile application: A prospective, observational study. Lancet Public Health 2021;6:e21–e29. Crossref, Medline, Google Scholar
- 33. Combining participatory influenza surveillance with modeling and forecasting: Three alternative approaches. JMIR Public Health Surveill 2017;3:e83. Crossref, Medline, Google Scholar
- 34. Veterans’ response to an automated text messaging protocol during the COVID-19 pandemic. J Am Med Inform Assoc 2020;27:1300–1305. Crossref, Medline, Google Scholar
- 35. Use of short message service in at-home COVID-19 patient management. BMC Med 2020;18:391. Crossref, Medline, Google Scholar .
- 36. COVID and CARE®. Mobile application for monitoring SARS-CoV-2 positive patients after hospitalization. Enferm Infecc Microbiol Clin 2021;39:261–262. Google Scholar .
- 37. Association between patient portal use and broadband access: A national evaluation. J Gen Intern Med 2020;35:3719–3720. Crossref, Medline, Google Scholar .
- 38. Disparities in telehealth use among California patients with limited English proficiency. Health Aff (Millwood) 2021;40:487–495. Crossref, Medline, Google Scholar .
- 39. The expanding digital divide: Digital health access inequities during the COVID-19 pandemic in New York City. J Urban Health 2021;98:183–186. Crossref, Medline, Google Scholar
- 40. A patient self-checkup app for COVID-19: Development and usage pattern analysis. J Med Internet Res 2020;22:e19665. Crossref, Medline, Google Scholar