Evaluating the Effect of a Telepsychiatry Educational Program on the Awareness, Knowledge, Attitude, and Skills of Telepsychiatry Among Spanish Psychiatrists during COVID-19 Pandemic

Introduction

Telepsychiatry (TP) is the branch of telemedicine that allows the delivery of mental health care from a distance through technology, often using videoconferencing, involving a range of services including psychiatric evaluations, therapy (individual therapy, group therapy, family therapy), patient education, and medication management.^1–3

Although TP has been working in one form or another for over 60 years, it has been gaining prominence since the 1990s, fully accepted by mental health professionals in the 2000s, and since then it has been considered as effective as face-to-face care, with clinical practice guidelines being available.⁴ TP facilitates access to medical care for those patients who live in rural areas or who have some difficulty of any kind in getting to the hospital.

In Spain, a growing number of health services based on telemedicine have been implemented in recent years as a complement (not substitute) to conventional services. However, the dissemination of this technology is still limited and slow, with many more projects being tested than finally implemented in the health sector.⁵ However, it has been necessary to wait until the onset of coronavirus disease (i.e., COVID-19 pandemic), for TP transformation from a niche practice to a standard way of psychiatric clinical practice.⁶ TP is here to stay as those services that have converted their face-to-face appointments into virtual appointments have found a number of benefits, including overall user satisfaction, reduced time wasted by eliminating travel, the possibility of continuing to receive care despite confinement and maintaining the economic activity of a clinic, advantage that the physician can be teleworking, and flexibility in choosing the best time to be seen.^7–9

The ultimate success of TP will depend on many factors including the knowledge and understanding of the concept, skills acquired, the attitude toward technology, greater number of hours of experience of clinicians, and working environment for psychiatrists.^10,11

The Spanish Society of Psychiatry is a private nonprofit organization established in 1958 that includes more than 1,200 Spanish psychiatrists. Its aim is to lead Spanish psychiatry; promote the study and development of psychiatry, and its scientific and technical progress, care, teaching, and research at all levels and areas of its competence; promote information and psychiatric training of the general practitioner and participate in the training of the specialist; and contribute, by advising and promoting research to improve the practice of psychiatry.

In this context, and taking into account the challenges in mental health care created by COVID-19 pandemic, characterized by calls for social distancing and concerns about the mental health effects of the pandemic, the Spanish Society of Psychiatry developed an online Practical Course of Telepsychiatry offered free of charge to its members with the aim of filling the educational gap in this matter among psychiatry professionals in Spain and promoting knowledge of the technology, methods and practical skills.

Since assessment needs to be part of an ongoing evaluation cycle intended to keep the curriculum fresh, educationally sound, and achieving its intended objectives, the main objective of this study was to assess the changes in awareness, knowledge, attitude, and willingness to accept and provide TP services among the Spanish psychiatrists after their participation in an online educational intervention of practical TP.¹²

Materials and Methods

STUDY SAMPLE AND DESIGN

A quasi-experimental online design with pre-intervention and immediate post-intervention test was used in this study, conducted from October to December 2020. Until the current COVID-19 pandemic, TP had hardly been used in routine clinical practice in Spain. The Spanish Society of Psychiatry developed a practical online TP course carried out by psychiatrist with expertise in the field with the aim of covering the educational gap among mental health professionals in Spain, reaching more than 80% of its members (1,277 psychiatrists) and providing them with training in TP and logistical support.

INTERVENTION

The online educative intervention comprised 22 teaching hours along 14 topics that included the evolution and history of TP and its impact on psychiatric care; its benefits and limitations in clinical practice; TP technical considerations; its legal, safety, and ethical aspects; the path to implementation; the practice of TP in rural/remote places; in primary care; in residencies for the elderly; in child and adolescent mental health care; clinical research in TP; and training and communication in TP practice.

INSTRUMENT

The telemedicine—awareness, knowledge, attitude, and skills (AKAS) questionnaire was originally designed and validated by Zayapragassarazan¹⁰ as a self-reported questionnaire consisting of six sections: (1) information about the personal and professional background of the respondent; (2) awareness level about telemedicine, through 12 statements with graded response on a three-point scale ranging from 0 to 2 that is, “0” for “don’t know,” “1” for “heard of it,” and “2” for “know about it”; (3) knowledge level with respect to telemedicine, through 11 statements to be answered in either “Yes” or “No,” giving a score of “1” for “Yes” and “0” for “No”; (4) attitude toward telemedicine, through 11 statements with a graded response on a five-point Likert scale ranging from 0 to 4 that is, “0” for strongly disagree, “1” for disagree, “2” for undecided, “3” for agree and “4” strongly agree; (5) skills in telemedicine, through 12 statements with a graded response to each statement on a four point scale ranging from 0 to 3 that is, “0” for “unskilled,” “1” for “learner,” “2” for “mediocre” and “3” for “expert”; and finally, (6) an open ended section that allowed the respondents to express their opinions and other comments related to the area of research. Our adapted version only changed the word telemedicine by TP. The completion of the questionnaire twice, before and after the educational intervention, was mandatory requirement to obtain the corresponding educational credits. All the participants signed an informed consent to participate in the project.

DATA ANALYSIS

Data collected were statistically analyzed using the Statistical Package for Social Sciences (SPSS, version 25.0). The raw scores were calculated for awareness, knowledge, attitude, and skills sections of AKAS. The normality of the entered data was checked by the Kolmogorov–Smirnov test and Fisher’s measures of skewness and kurtosis. Mean and standard deviation (SD) were used to describe and summarize continuous variables while frequency expressed in percentages was used for categorical variables. Mean scores of AKAS were computed at the pre-intervention and at the end of educative intervention (post-intervention). Raw scores for the AKAS sections were converted to percentage considering that the scores equal to and <49% were considered as low with respect to AKAS, the scores between 50% and 70% were considered as average and the scores equal to and above 71% were considered as high. The change of scores and percentages between pre and post-intervention in AKAS sections analyzed was investigated using Chi-square test, paired t-, and analysis of variance (ANOVA) test with statistically significant level set at p ≤ 0.05.

Results

A total of 213 psychiatrists enrolled in the online course of practical TP and filled pre-intervention AKAS questionnaires. Of these, 152 completed the course and filled the post-intervention questionnaires. Table 1 shows the distribution of participating psychiatrists according to their sociodemographic variables. The sample initially consisted mainly of women (59%), younger than 40 years (61.9%), working in public sector (71.4%), in hospitals without TP units (81.7%), with a mediocre computer and internet knowledge (67.6%), and never attended any training course in TP (87.8%).

Along Tables 2–4 the results on the AKAS questionnaire are depicted. Table 2 shows mean and SD for questionnaire sections scores of the overall sample and sub-samples before and after the educative intervention. At pre-intervention, it should be noted that female participants self-reported significant lower scores in skills (F = 4,758, p = 0.030), older participants self-reported higher values in awareness score (F = 6,192, p = 0.0001), and psychiatry residents self-reported significant lower values of awareness than psychiatrists (F = 9,358, p = 0.003). In Table 3 the levels of AKAS among the sample before and after the educational intervention are described.

At pre-intervention, the awareness level shows that 12% of the participants have low level of awareness, 17% have average level, and 71% have high level; the knowledge level shows that only 2% of the participants have low level of knowledge of TP, 37% have average, and 41% have high level; concerning the attitude level, 5% of the respondents have low level of attitude toward TP, 33% possess an average level and 62% possess high level; and, according to the skill of TP, 14% do not have adequate skills, 29% possess average skills, and 57% have adequate skills. Table 4 shows the effect of the telepsychiatry educational intervention on the results of the AKAS questionnaire. There was a statistically significant increase in the awareness, knowledge, and attitudes scores but no in skills about TP. Figure 1 shows box plot graphics with pre- and post-AKAS questionnaire sections values where it is possible to demonstrate the improvement among the participants, especially in the first two scales (Awareness and Knowledge).