Skip to main content

A mixed methods evaluation of the feasibility, acceptability, and impact of a pilot project ECHO for community health workers (CHWs)

Abstract

Background

Despite the positive effects of community health workers (CHWs) on addressing social determinants of health, improving patient health outcomes, and decreasing overall healthcare costs, there is a lack of standardization in training and certifying this workforce, resulting in different approaches to integrating this role into medical home models. The purpose of the current study is to evaluate the application of Project ECHO (Extension for Community Healthcare Outcomes) in enhancing CHWs’ capacity to address health and social issues of vulnerable populations.

Methods

An explanatory sequential mixed methods design was applied in which all participants (N = 49) completed pre (January 2019) and post (July 2019) quantitative online surveys measuring changes in self-efficacy, behavior change intent, and knowledge. Virtual focus groups were conducted with a subset of participants (n = 20) in July 2019 to assess the feasibility, acceptability, and impact of Project ECHO.

Results

There was a statistically significant difference of + 0.453 in the composite self-efficacy mean score pre- to post-series. For every 1 additional Project ECHO CHW session attended, there was a .05 improvement in participants’ self-efficacy to perform CHW-related job duties and address social determinants of health (SDOH). Four major themes emerged from the qualitative focus group data: value in learning from other participants’ caseloads, CHW-care team integration, availability of training and resources, and shared decision-making with patients.

Conclusions

This evaluation suggests that ECHO is a viable means of increasing access to training resources for CHWs. Future studies on the ECHO model as a means of educating and broadening implementation of CHWs are warranted. Programs such as Project ECHO can support CHWs by providing continuing education opportunities, as well as standardizing training content across large geographic areas.

Peer Review reports

Key messages regarding feasibility

  • This evaluation suggests that Project ECHO can be a feasible and acceptable tool to provide continuing education and training for community health workers (CHWs). Participants shared how certain skills and knowledge obtained from ECHO CHW could be deployed in their respective communities and healthcare settings. Knowledge and self-efficacy gains in didactic areas such as integrating trauma-informed care, enacting motivational interviewing skills, and changing patients’ health behaviors indicated its impact.

  • The participants’ positive regard of the curriculum topics, didactic presentations, and case recommendations demonstrated acceptability. In acknowledging that the video conferencing environment was conducive to their learning needs, ECHO CHW participants provided context for the initiative’s overall feasibility.

  • The current study’s feasibility findings suggest that Project ECHO model can play an important role in supporting and providing CHWs with continuing education to help address challenges faced by patients beyond the traditional domain of healthcare provision. Programs like ours that train and support CHWs are increasingly important as health systems, prompted in large measure by changes in payment models, seek new strategies to improve quality and value.

Introduction and background

Health initiatives that target social determinants of health (SDOH) underscore the notion that a patient’s environment influences their overall health and well-being. Quality improvement efforts in healthcare have traditionally focused on ameliorating poor health indicators within clinical practice [1]. Although this approach may lead to improvements in some health outcomes, it often fails to account for the significant influence of health-related social factors such as housing, access to food, transportation, and employment.

Interventions that address SDOH are based on the emerging consensus that improved health outcomes cannot be achieved by the healthcare sector alone. Rather, collaboration among healthcare providers, patients, and patients’ natural supports (e.g., family, friends, community members) is necessary to improve health and promote wellness. Federal initiatives, such as Healthy People 2020, have established national objectives to create environments that promote good health for all [2]. Initiatives like these identify the most pressing preventable threats to health and establish indicators to diminish these threats. These initiatives target local communities and emphasize whole-person-centered healthcare systems supported by programs such as Community-Clinical Integration Programs and Patient-Centered Medical Homes [3, 4]. Efforts to improve healthcare access, lower healthcare costs, and improve population health are at the crux of these initiatives and underscore the role SDOH play in population health and well-being.

One workforce functioning at the intersection of healthcare and community is community health workers (CHWs). The American Public Health Association defines a CHW as the following:

A frontline public health worker who is a trusted member of and/or has an unusually close understanding of the community served. This trusting relationship enables the worker to serve as a liaison/link/intermediary between health/social services and the community to facilitate access to services and improve the quality and cultural competence of service delivery [5].

Fostering trust between community residents and healthcare systems is not a new concept. For more than 50 years, CHWs have worked to address healthcare concerns, expand access to preventive health services, and mitigate health disparities in historically vulnerable and medically underserved communities [6]. CHWs bridge language, literacy, and cultural meanings for people and communities while promoting health education, care coordination, and linkages to resources [7,8,9]. By assuming a broad range of responsibilities, CHWs bridge the gap between primary care provision and well-being outside the clinical setting, particularly among patients most in need. Several studies have demonstrated reductions in care utilization and more equitable access to curative services for vulnerable patients managing chronic health conditions when CHWs were included in the patient’s care team ([10,11,12,13]; Moffett M, Kaufman A, & Bazemore, 2019 [14];). One randomized control trial of CHWs supporting low-income patients with multiple chronic conditions demonstrated improvements in health outcomes, mental health, and reductions in hospitalizations [7]. Similarly, a systematic review revealed that CHW interventions can significantly reduce emergency room visits, hospitalizations, and urgent care visits among patients served in various healthcare settings [12].

Despite the positive effects of CHWs on patient health outcomes and overall healthcare costs, there is a lack of standardization in training and certifying this workforce, resulting in different approaches to integrating this role into medical home models. A national survey of the CHW workforce revealed variations in role definition and training criteria to support their professional development [12]. Online and in-person training resources for CHWs also vary considerably in content, quality, and access [15,16,17].

One promising approach to addressing this gap is Project Extension for Community Healthcare Outcomes (ECHO). Project ECHO is a novel educational intervention that uses case-based learning and video conferencing to connect experts in a particular discipline with frontline healthcare workers. Since 2003, ECHO has been used to connect primary care providers in medically underserved environments with ECHO specialists in order to supplement knowledge and enhance self-efficacy regarding treatment of specific diseases [18]. While there is emerging evidence that Project ECHO has a positive impact on provider behavior and self-efficacy, less is known about its utility as a tool to support non-medical provider members of the healthcare team such as CHWs. Few studies [17, 19] to date have evaluated the use of the ECHO model for CHWs. Moreover, prior applications of ECHO to educate CHWs focused on specific health conditions or diseases and were often designed for CHWs within narrow geographic regions. In this paper, we examine the feasibility, acceptability, and impact of a national Project ECHO training program created specifically for CHWs to enhance their competency and improve their ability to address patients’ SDOH. Prior implementation research related to Project ECHO has identified feasibility and acceptability as key components to ensuring implementation success [20], yet it is unknown to what extent CHWs will choose to participate in and have their learning needs met by a national ECHO with a broader curriculum. In addition to feasibility and acceptability, we also assess impact so as to understand immediate outcomes related to the intervention. To our knowledge, this is the first mixed methods evaluation of a national, innovative application of ECHO to train CHWs in addressing the health and social needs of their respective patient populations.

Methods

Project ECHO CHW intervention and sample

In an effort to support standardization and sustainability of training for CHWs, and with funding from the Center for Medicare and Medicaid Innovation State Innovation Model (SIM), the Weitzman Institute, a research organization embedded in a Federally Qualified Health Center (FQHC), partnered with the Penn Center for CHWs to pilot the implementation of Project ECHO for community health workers (ECHO CHW). Project ECHO CHW was a 6-month virtual, case-based learning initiative designed to enhance participants’ capacity to address health issues for vulnerable populations typically served by CHWs.

This ECHO series connected CHWs and those in similar roles, including hospital-based patient care coordinators, case managers, and patient advocates, with faculty mentors from the Penn Center for CHWs and Community Health Center, Inc. While specifically designed for CHWs, Project ECHO CHW program implementation staff permitted healthcare workers who facilitated healthcare and social service system navigation to engage in the series alongside their CHW peers. This enabled interdisciplinary dialogue among healthcare workers serving in similar capacities to the traditional CHW. The initiative had four distinct, but related goals: (1) to enhance participants’ knowledge of general CHW core competencies, (2) to augment participants’ capacity to address SDOH, (3) to assist participants’ in their ability to work alongside patients in the attainment of specified health goals, and (4) to build a national learning community where CHWs can receive support in working with their challenging patient cases, share resources with other participants regionally and nationally, and learn from an expert faculty panel.

Faculty and curriculum

Faculty members, including one senior CHW, an assistant director of training for CHWs, a patient navigator, a program manager, and a nurse manager, were selected based on their expertise in the areas of case management, clinical care, community referral processes, and training CHWs. Learning sessions were 90 min and held twice monthly by video conference from January 2019 through July 2019. Faculty used approximately 30 min to present didactic lectures on curriculum topics such as qualitative interviewing, trauma-informed care, and connecting patients to community resources. The list of didactic topics covered during the ECHO CHW intervention is included in Appendix B. The remaining hour was devoted to case-based learning, which involved participants presenting details about the biopsychosocial challenges of patients they were supporting, and faculty leading a discussion and sharing recommendations for each particular case.

Participant recruitment

The intervention was open to CHWs nationwide. Recruitment was conducted via email utilizing a national list of contacts primarily from FQHCs and other primary care practices and word of mouth within Connecticut. A total of 120 participants from 33 organizations across 19 states and the District of Columbia enrolled in the program. All participants were over 18 years of age and English speaking. The Institutional Review Board at the Community Health Center, Inc. approved all study procedures.

Pre-post design

Pre- and post-surveys were developed through a collaborative, iterative approach between the research staff and program implementation team. Research and program staff developed a set of questions to gauge participants’ confidence to deploy skills related to certain didactic topics outlined in the initiative’s curriculum (e.g., How confident are you that you can recognize a patient’s motivation to adopt a particular health behavior? How confident are you that you can connect a patient with community resources that meet that person’s needs?). The survey was a 35-question instrument that captured participants’ roles, years of experience, prior training, age, race/ethnicity, gender identity, and responses to self-efficacy questions (20 items) aligned with 8 of the 12 ECHO CHW didactic topics. Participants completed the Participant Enrollment Form (baseline, pre-survey) prior to joining any of the 12 scheduled ECHO CHW sessions. The link to the post-series survey included the same set of self-efficacy questions (20 items) and was sent via email following the last ECHO CHW session to everyone who completed a Participant Enrollment Form and participated in at least one ECHO session. For data quality and management purposes, ECHO CHW participants were assigned a unique participant code at the beginning of the learning series. Participants input their assigned participant code to access the baseline and post-series surveys, which enabled us to match pre- and post-surveys for data analysis. Additionally, a mid-series survey assessed participants’ satisfaction with various elements of the ECHO program in addition to self-reported behavior changes as a result of engagement in the intervention.

Semi-structured focus groups

Two separate, 1-h focus groups with ECHO CHW participants were conducted at the conclusion of the program. The focus group questions sought to determine participants’ perceptions about the feasibility, acceptability, and impact of the program. A total of twenty participants took part in both focus groups. The semi-structured interview guide is included in Appendix C.

Measurement strategy

This evaluation applied a sequential explanatory mixed methods design [21] in which open-ended focus group questions are used to provide explanation and context for survey responses (Damian, Gallo, Leaf & Mendelson, [22]). More specifically, the five parts of the focus group questions (Appendix C) contextualized the survey questions, which also primarily focused on examining content, attendance, resources, engagement and application, and support and relationships from the participants’ perspectives. Additionally, this evaluation’s methodology was guided by Moore’s Model of Outcomes Assessment framework, which has been applied to a range of other ECHO clinics (Zhou, Crawford, Serhal, Kurdyak & Sockalingam, [23]). The quantitative survey questions, in addition to semi-structured interview questions, align with levels 1 through 4 of Moore’s assessment framework, which is referenced in Appendix A.

Moore’s model of outcomes assessment level 1: participation

The Project ECHO program coordinator maintained program attendance records for the entire duration of the intervention through Zoom video conference reporting. Attendance confirmation emails were also sent immediately following each session due to limitations in Zoom capturing participants that join from the same computer or via telephone.

Moore’s model of outcomes assessment level 2: satisfaction

A mid-series survey was administered in April 2019 to assess participants’ satisfaction with and acceptability of this ECHO program. The survey included 11 satisfaction-related items on a 5-point Likert scale (1 = very dissatisfied and 5 = very Satisfied) and 6 items on a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree) regarding the extent to which Project ECHO met participants’ learning needs. Participants also provided open-ended feedback regarding their overall satisfaction with the session’s didactic lectures, case presentations, and faculty recommendations.

Moore’s model of outcomes assessment level 3A: knowledge

The ECHO CHW faculty developed two to three multiple choice knowledge poll questions specific to each didactic session. These questions were delivered through a polling feature on the video conferencing platform at the beginning (pre-) and end (post-) of didactic sessions 5 through 12.

Moore’s model of outcomes assessment level 3B: self-efficacy

The 20-item self-efficacy survey measured participants’ belief in their ability to perform core competencies related to the didactic sessions using a 4-point Likert scale (1 = not at all confident, 4 = very confident). Higher mean scores indicated greater degree of confidence in participants’ ability to perform duties related to the CHW core competencies.

Moore’s model of outcomes assessment level 4: competence (behavior change intent)

Behavioral change refers to one’s attitude about the likelihood to engage in a behavior at a specific time and place. The mid-series survey included three qualitative questions that assessed participants’ intent to enact behaviors aligned with the case-based learning and didactics they had been exposed to up to the midway point in the entire series (e.g., Is there anything you plan to do differently in your professional role based on knowledge you gained from Project ECHO CHW? Please describe how you plan to apply what you learned.). Semi-structured focus group interviews conducted at the conclusion of the intervention and described in greater detail below included two qualitative questions regarding participants’ intent to change behavior based on lessons learned from the intervention.

Other covariates

Research staff also collected information about sociodemographic characteristics such as gender (male, female, not specified), professional role (community health worker, care coordinator, case manager, patient navigator), years of experience in role (< 1 year, 1–4 years, 5–9 years, 10–15 years, > 15 years), and prior training in CHW core competencies (yes/no).

Analysis strategy

Descriptive and inferential statistical analyses of quantitative survey data were performed in IBM SPSS 22 and STATA 15. To measure internal consistency, an inter-item correlation was performed on the 20-item self-efficacy pre-post survey. The alpha coefficient for the twenty items was 0.93, suggesting that the survey had relatively high internal consistency. The alpha coefficients for subscales within the survey pertaining to the eight of twelve total didactic topics, however, varied. The separate subscales reflected eight of the twelve didactic topics outlined in the ECHO CHW curriculum and were comprised of two to three items each. The trauma-informed care subscale consisted of 3 items (α = 0.77), the mental health subscale consisted of 2 items (α = 0.81), the health behaviors subscale consisted of 3 items (α = 0.90), the re-engaging patients subscale consisted of 2 items (α = 0.27), the conflict resolution subscale consisted of 3 items (α = 0.73), the confidentiality subscale consisted of 2 items (α = 0.40), and the ending the patient relationship subscale consisted of 2 items (α = 0.92). The survey only contained one question on motivational interviewing. A total of 119 participants enrolled to participate in ECHO CHW and responded to pre-survey measuring self-efficacy. Five of these participants notified the program staff of their desire to discontinue their participation prior to the end of the learning series, and eight additional participants registered, however, never attended any of the sessions. Fifty-one participants completed the post-survey at the conclusion of the series, which resulted in a 48% response rate. As such, a latent change score model was used to calculate the extent to which changes from pre to post were significant when accounting for missing post-survey data due to pre-post survey completion attrition. The latent change score model created true change values in self-efficacy without ignoring missing data at the post-series marker. All descriptive and inferential tests of survey data were performed in STATA 15.

Open-ended survey responses and focus group data were analyzed qualitatively. Open-ended survey feedback was considered in the context of this evaluation’s aims and analyzed accordingly. Focus group data were transcribed from a video recording, de-identified for confidentiality, and reviewed independently by two trained coders from the research team. The research team read through each transcript three times, coding key statements. The iterative coding process that involved clustering of keywords was followed by thematic analysis, whereby a search for themes that emerged in the dataset occurred. Thematic analysis involves the identification of themes through careful reading and re-reading of the data [24]. A third trained coder from the research team was included to assist with developing consensus around themes and address any disagreements between the two initial coders. NVivo 12 Plus was used to organize and analyze focus group data.

Results

Sample characteristics

Table 1 shows the demographics of the analytic sample (N = 51). While about half of the participants (49%) identified as CHWs, a third of participants (33.3%) selected “Other (not listed),” indicating that they either held a different title but engaged in similar roles (e.g., recovery navigator), or supervised CHWs (e.g., directors, program managers). A small proportion (18%) of participants had less than a year of experience in their current role. Most participants were female (94%) and had prior training in CHW core competencies (55%).

Table 1 Demographics of analytical sample in Project ECHO community health worker training (N = 51)

Attendance

The majority of participants (76.4%) attended at least half of the sessions, with 22 (43.1%) participants attending 50–79% of sessions and 26 (16.6%) participants attending > 80% of sessions.

Satisfaction

Fifty out of the 51 individuals in the analytical sample completed the mid-series satisfaction questionnaire. Table 2 shows results of satisfaction ratings for 11 areas. With the exception of one topic area (technical assistance from Project ECHO staff), over 90% of participants indicated that they were very satisfied/satisfied with each of the noted satisfaction areas. About a quarter (22%) of participants noted that technical assistance from Project ECHO staff was not applicable and therefore could not provide a satisfaction rating. Nonetheless, while participants found the intervention favorable overall, 1 out of 10 participants was dissatisfied/very dissatisfied with the duration of sessions, 8% were dissatisfied/very dissatisfied with engagement with other participants, and 6% were dissatisfied/very dissatisfied with the frequency of the sessions.

Table 2 Results from mid-series satisfaction survey (n = 50)

Changes in knowledge

Table 3 shows results from the pre- and post-knowledge poll questions from didactic sessions 5 through 7, and 9 through 12 (seven didactic sessions total). The knowledge poll data captured during the 8th session were invalid due to a technical glitch which resulted in the wrong pre-question presented during the didactic session. Overall, there was an average of 10% improvement in participants’ knowledge.

Table 3 Changes in participants’ knowledge (the knowledge poll data captured during the 8th session were invalid due to a technical glitch which resulted in the wrong pre-question being presented during the didactic session)

Changes in self-efficacy

A paired sample t test was performed to compare self-efficacy scores among the 51 participants before and after the ECHO CHW initiative (Table 4). There was a statistically significant difference of + 0.453 when comparing self-efficacy scores before the initiative (M = 3.14, SD = .56) and after the initiative (M = 3.60, SD = .35; t(50) = 6.512, p < .001). The latent change score model calculation revealed a dose-response relationship between number of ECHO CHW sessions attended and improvements in self-efficacy. In other words, those who attended more ECHO CHW sessions showed a larger increase in self-efficacy. More specifically, for every 1 additional session attended, there was a .05 improvement in self-efficacy (or rather, 5 more sessions attended led to .25 larger change in self-efficacy). The latent change score model accounted for missing data due to attrition in survey completion pre- to post-engagement in ECHO CHW. A total of 119 participants completed a baseline, pre-series Enrollment Form; however, 51 completed the post-series survey. As such, we calculated a latent change score to create true change values in self-efficacy from pre to post without ignoring missing data at the post-series marker.

Table 4 Changes in self-efficacy (statistical significance detected if p < .05)

Behavior change intent

With regard to the three questions on behavior change intent that were included in the mid-series survey, three out of five (63%) participants endorsed the intention of approaching their work differently based on knowledge gained from Project ECHO, and four out of five (80%) participants indicated that they did not foresee any barriers to implementing the intended behavior changes. Over half (55%) of the participants, however, noted that they had not yet made any changes based on knowledge gained from Project ECHO.

Participants’ perspectives on the program

Four major themes emerged from review of the qualitative data (Table 5). Two themes related to the usefulness of the ECHO CHW intervention, and two related to CHWs in the context of the healthcare team.

Table 5 Themes identified through qualitative analysis of participant interviews

Themes related to the usefulness of the ECHO CHW intervention. Two themes related to the usefulness of the ECHO CHW intervention that were constructed from the responses included the intervention as a viable education tool for supporting CHW programming and heightened intent to improve patient-centered and team-based care practices. Participants endorsed ECHO CHW as convenient due to the use of the video conference platform. Moreover, participants described feeling supported by being able to connect with peers from practices across the country who have shared challenges while learning about cases that they had not encountered before. Additionally, participants noted the program heightened their intent to improve patient-centered and team-based care practices, including feeling better positioned to engage patients in shared decision-making. Participants also noted their intent to use skills learned from the sessions to better support, leverage, and improve competencies of colleagues.

Themes related to CHWs in the context of the healthcare team. Two themes related to CHWs in the context of the healthcare team that were observed from participants’ responses included variability in roles and responsibilities of CHWs and challenges in validating CHW’s added value to care team. Several participants noted great variation in the responsibilities of CHWs within and across practice sites. For example, some practices engaged CHWs in screening and referral for health-related social needs (e.g., transportation), whereas others had CHWs more involved with chronic disease management. Nonetheless, some participants reported having a difficult time gaining the trust of colleagues due to reservations regarding CHWs’ added value and ability to assist with patients’ health and social needs. One participant attributed these challenges to policy-related factors, including lack of financial incentives and reimbursement structures to support inclusion of CHWs as part of healthcare teams.

Triangulation of data from both the pre-post surveys and focus groups demonstrated alignment regarding faculty satisfaction as a measure of acceptability, with 98% of the participants rating their satisfaction with faculty favorably. Several participants mentioned the importance of having a senior CHW as part of the faculty, suggesting that a senior CHW provides an opportunity for participants to learn from experienced professionals who have “traveled a similar road” and can advise on best practices. Additionally, both quantitative and qualitative findings suggest that participants’ self-efficacy significantly increased as a result of participation. Those with higher levels of engagement (as measured by number of sessions attended) experienced greater increases in self-efficacy from pre- to post-intervention. Focus group participants also shared that they had already modified their behavior based on information learned from the ECHO program or planned to do so in the future.

Nonetheless, not all aspects of the program were viewed favorably. Four out of five participants (80%) were dissatisfied or very dissatisfied with the ability to engage with other participants during the ECHO sessions. Focus group findings corroborated this finding. One participant shared that having other participants’ contact information would enhance interaction by enabling participants to communicate directly with each other during the sessions.

Discussion and conclusion

This evaluation suggests that Project ECHO can be a useful tool to provide continuing education and training for CHWs. CHWs are increasingly being assigned to healthcare teams to help improve patient outcomes based on an increasing recognition of the critical role played by SDOH. These issues, while traditionally not emphasized in medical care, may be the key to reducing health inequality and improving the health and well-being of the broader population. Moreover, the challenge some participants faced gaining trust and validation of the importance of their role from the care team suggests that additional work is needed to educate front line clinical providers on the impact of SDOH and the need to address them as part of clinical care.

Through the use of mixed quantitative and qualitative methods, ECHO CHW participants provided robust perspective about the feasibility, acceptability, and impact of the initiative. Participants shared how certain skills and knowledge obtained from ECHO CHW could be deployed in their respective communities and healthcare settings. Knowledge and self-efficacy gains in didactic areas such as integrating trauma-informed care, enacting motivational interviewing skills, and changing patients’ health behaviors indicated its impact. The participants’ positive regard of the curriculum topics, didactic presentations, and case recommendations demonstrated acceptability. In acknowledging that the video conferencing environment was conducive to their learning needs, ECHO CHW participants provided context for the initiative’s overall feasibility.

Through attendance tracking and qualitative interviews with ECHO CHW participants, we learned about the convenience of connecting learners through the ECHO CHW video conferencing platform. This accessibility lent well to the feasibility of the pilot program. Participants spanning a wide geographic area connected with peers in order to share how they resolve SDOH-related issues in their patients’ lives. The project team anticipated that many of those who registered to participate in ECHO CHW would continue throughout the duration of the pilot program. Data show that of the 114 participants who remained actively enrolled in ECHO CHW, 8 participants (7.0%) did not attend any sessions as compared to 66 (54.4%) who attended at least 50% of the ECHO CHW sessions. While not conclusive, these attendance rates in addition to participants’ qualitative responses about how they would deploy content learned from ECHO help us better understand how feasible it was for participants to attend sessions of interest to them.

This study was unique in two important ways. First, the ECHO intervention was not focused on a specific disease or condition. Rather, this intervention provided continuing education and training more broadly and used cases to support and reinforce role-specific core competencies rather than to provide upskilling in a more specialized area. Though initially developed to improve access to evidence-based treatment for hepatitis C (Arora, Kalishman, & Thornton, [25]), subsequent ECHO programs have addressed other conditions such as substance abuse, pain care, diabetes, and hypertension ( [26]; Bouchonville, Hagar, Kirk, Qualls, & Arora, [18]; Katzman, Comerci, & Landen, [27]; Katzman, Fore, & Bhatt, [28]; Khatri, Haddad, & Anderson, [29]; Komaromy, Duhigg, & Metcalf A, [30]). While evidence is growing to suggest that disease-focused ECHO programs improve clinical outcomes (Anderson, Zlateva, Davis & Spegman, [31]; Bouchonville, Hagar, Kirk, Qualls, & Arora, [18], Kawasaki, [32]; Komaromy, Bartlett, Manis, & Arora, [33]; Korthuis, McCarty, & Weimer, [34]; Thies, Anderson, & Beals-Reid, [35]), less is known about the model’s efficacy to provide broader training not focused on a particular illness. Similarly, although one prior study evaluated the use of Project ECHO to train CHWs, it was focused specifically on diabetes (Zurawski, Komaromy, Ceballos, McAuley, & Arora, [17]).

Secondly, this intervention was unique in that the intended audience was comprised of CHWs. Most ECHO programs have been designed with medical providers in mind; however, less is known about the impact and utility of ECHO for other members of the healthcare team. In this study, CHWs were actively engaged participants. In addition to their high degree of satisfaction, participants demonstrated significant improvements in knowledge and behavior change intent, suggesting that participants, as they reviewed and discussed CHW core competencies during the didactics, were gaining confidence building skills that they would apply to their work.

Nonetheless, this study has several notable limitations. While we demonstrated increase in knowledge for some sessions, the overall response rate for completing the pre-post knowledge assessment was low (42%). The pre-session knowledge poll question was administered at the outset of each ECHO session. Unfortunately, not all attendees logged in on time, leading to low rates of survey completion. Moreover, knowledge assessment was limited to what was taught in the didactics part of each session. A broader pre-post knowledge assessment of CHW skills would have provided a useful addition to the topic-specific assessments conducted at each session. Lastly, while we demonstrated an increase in intent to apply new knowledge, additional studies are needed to more definitively assess whether these preliminary positive findings translate into measurable improvements in patient outcomes.

With these limitations in mind, we conclude that the Project ECHO model can play an important role in supporting and providing CHWs with continuing education to help address challenges faced by patients beyond the traditional domain of healthcare provision. Programs like ours that train and support CHWs are increasingly important as health systems, prompted in large measure by changes in payment models, seek new strategies to improve quality and value. Value-based payment models, with their emphasis on non-visit-related reimbursement, provide a powerful incentive for health systems to invest in interventions aimed at improving not only clinical process measures within the care delivery system, but larger, population-based measures as well. With increasing recognition of the impact of environmental and other factors on health outcomes, interventions such as CHWs can play an essential role in achieving clinical and financial goals for primary care practices and larger health systems. Project ECHO is a scalable tool with minimal technology requirements that nearly anyone in any location can access. Moreover, increasing recognition of the importance of CHWs has led many states to establish CHW certification requirements (London, Carey, & Russell, [36]) including requirements for continuing education. Programs such as ours can support CHWs in meeting these requirements and standardize training content across large geographic areas. Reducing health inequality will require a concerted emphasis on addressing SDOH, which CHWs are ideally suited to support. Project ECHO for CHWs can help ensure that this much-needed role is supported with evidence-based guidance and peer support and interaction.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CHW:

Community health worker

ECHO:

Extension for Community Healthcare Outcomes

FQHC:

Federally Qualified Health Center

SDOH:

Social determinants of health

SIM:

State Innovation Model

References

  1. 1.

    Mold J, Peterson K. Primary care practice-based research networks: working at the interface between research and quality improvement. Ann Fam Med. 2005;3:S12–20.

    Article  Google Scholar 

  2. 2.

    U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion. (n.d.). Healthy people 2020. Retrieved November 1, 2019 from https://www.healthypeople.gov/2020/topics-objectives.

  3. 3.

    Moffett ML, Kaufman A, Bazemore A. Community health workers bring cost savings to patient-centered medical homes. J Community Health. 2017;43:1–3.

    Article  Google Scholar 

  4. 4.

    Zahn, D., Corrado, M., & Rubin J. (2019). Making community and clinical integration work: a guide for moving from idea to implementation. Retrieved November 11, 2019 from https://www.cthealth.org/wp-content/uploads/2019/07/CHF-CCI-Report-3.1-pages.pdf.

  5. 5.

    American Public Health Association (n.d.). Community health workers. Retrieved December 20, 2019 from https://www.apha.org/apha-communities/member-sections/community-health-workers.

  6. 6.

    Witmer A, Seifer SD, Finocchio L, Leslie J, O’Neil EH. Community health workers: integral members of the health care work force. Am J Public Health. 1995;85:1055–8.

    CAS  Article  Google Scholar 

  7. 7.

    Kangovi S, Mitra N, Grande D, Huo H, Smith R, Long J. Community health worker support for disadvantaged patients with multiple chronic diseases: a randomized clinical trial. Am J Public Health. 2017;107:1660–7.

    Article  Google Scholar 

  8. 8.

    Love M, Gardner K, Legion V. Community health workers: who they are and what they do. Health Educ Behav. 1997;24:510–22.

    CAS  Article  Google Scholar 

  9. 9.

    Roman LA, Raffo JE, Meghea CI. Maternal perceptions of help from home visits by nurse–community health worker teams. Am J Public Health. 2012;102:643–5.

    Article  Google Scholar 

  10. 10.

    Campbell J, Brooks M, Hosokawa P, Robinson J, Song L, Krieger J. Community health worker home visits for Medicaid-enrolled children with asthma: effects on asthma outcomes and costs. Am J Public Health. 2015;105:2366–72.

    Article  Google Scholar 

  11. 11.

    Ingram M. Community health worker impact on chronic disease outcomes within primary care examined using electronic health records. Am J Public Health. 2017;107:1668–74.

    Article  Google Scholar 

  12. 12.

    Jack HE, Arabadjis SD, Sun L, Sullivan EE, Phillips RS. Impact of community health workers on use of healthcare services in the United States: a systematic review. J Gen Intern Med. 2016;32:325–44.

    Article  Google Scholar 

  13. 13.

    Kim K, Choi JS, Choi E. Effects of community-based health worker interventions to improve chronic disease management and care among vulnerable populations: a systematic review. Am J Public Health. 2016;106:e3–e28.

    Article  Google Scholar 

  14. 14.

    Rothschild SK. Mexican American trial of community health workers: a randomized controlled trial of a community health worker intervention for Mexican Americans with type 2 diabetes mellitus. Am J Public Health. 2014;104:1540–8.

    Article  Google Scholar 

  15. 15.

    Covert, H., Sherman, M., Miner, K., & Lichtveld, M. (2019). Core competencies and a workforce framework for community health workers: a model for advancing the profession. Am J Public Health, 109, 320-327.

  16. 16.

    Sabo S, Allen CG, Sutkowi K, Wennerstrom A. Community health workers in the United States: challenges in identifying, surveying, and supporting the workforce. Am J Public Health. 2017;107:1964–9.

    Article  Google Scholar 

  17. 17.

    Zurawski A, Komaromy M, Ceballos V, McAuley C, Arora S. Project ECHO brings innovation to community health worker training and support. J Health Care Poor Underserved. 2016;27:53–61.

    Article  Google Scholar 

  18. 18.

    Bouchonville, M.F., Hagar, B.W., Kirk, J.B., Qualls, C.R., & Arora, S (2018). Endo ECHO improves primary care provider and community health worker self-efficacy in complex diabetes management in medically underserved communities. Endocr Pract, 24, 40-46.

  19. 19.

    Komaromy M, Ceballos V, Zurawski A, Bodenheimer T, Thom D, Arora S. Extension for community healthcare outcomes (ECHO): a new model for community health worker training and support. J Public Health Policy. 2018;39:203–16.

    Article  Google Scholar 

  20. 20.

    Serhal E, Arena A, Sockalingam S, Mohri L, Crawford A. Adapting the consolidated framework for implementation research to create organizational readiness and implementation tools for project ECHO. J Contin Educ Heal Prof. 2018;38:145–51.

    Article  Google Scholar 

  21. 21.

    Gallo JJ, Lee SY. Mixed methods in behavioral intervention research. In: Gitlin LN, Czaja SJ, editors. Behavioral intervention research. New York, New York, NY: Springer; 2016. p. 195–211.

    Google Scholar 

  22. 22.

    Damian AJ, Gallo JJ, Leaf P, Mendelson T. Organizational and provider level factors in implementation of trauma-informed care after a city-wide training: an explanatory mixed methods assessment. BMC Health Serv Res. 2017;17:750.

    Article  Google Scholar 

  23. 23.

    Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The impact of project ECHO on participant and patient outcomes: a systematic review. Acad Med. 2016;91:1439–61.

    Article  Google Scholar 

  24. 24.

    Rice PL, Ezzy D. Qualitative research methods: a health focus. South Melbourne, Australia: Oxford University Press; 1999.

    Google Scholar 

  25. 25.

    Arora S, Kalishman S, Thornton K. Expanding access to hepatitis C virus treatment-- extension for community healthcare outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology. 2010;52:1124–33.

    Article  Google Scholar 

  26. 26.

    Arora S, Thornton K, Murata G, Deming P, Kalishman S, Dion D, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. 2011;364:2199–207.

    CAS  Article  Google Scholar 

  27. 27.

    Katzman JG, Comerci GD, Landen M. Rules and values: a coordinated regulatory and educational approach to the public health crises of chronic pain and addiction. Am J Public Health. 2014;104:1356–62.

    Article  Google Scholar 

  28. 28.

    Katzman JG, Fore C, Bhatt S. Evaluation of American Indian Health Service training in pain management and opioid substance use disorder, American journal of public health. 2016;106:1427–9.

  29. 29.

    Khatri K, Haddad M, Anderson D. Project ECHO: replicating a novel model to enhance access to hepatitis C care in a community health center. J Health Care Poor Underserved. 2013;24:850–8.

    Article  Google Scholar 

  30. 30.

    Komaromy M, Duhigg D, Metcalf A. Project ECHO (extension for community healthcare outcomes): a new model for educating primary care providers about treatment of substance use disorders. Subst Abus. 2016;37:20–4.

    Article  Google Scholar 

  31. 31.

    Anderson D, Zlateva I, Davis B, Spegman D. Improving pain care with Project ECHO in community health centers. Pain Medicine. 2017;18:1882–9.

    Article  Google Scholar 

  32. 32.

    Kawasaki S, Francis E, Mills S, Buchberger G, Hogentogler R, Kraschnewski J. Multimodel implementation of evidence-based care in the treatment of opioid use disorder in Pennsylvania. J Subst Abus Treat. 2019;106:58–64.

    Article  Google Scholar 

  33. 33.

    Komaromy M, Bartlett J, Manis K, Arora S. Enhanced primary care treatment of behavioral disorders with ECHO case-based learning. Psychiatr Serv. 2017;68:873–5.

    Article  Google Scholar 

  34. 34.

    Korthuis PT, McCarty D, Weimer M. Primary care-based models for the treatment of opioid use disorder: a scoping review. Ann Intern Med. 2017;166:268–78.

    Article  Google Scholar 

  35. 35.

    Thies, K., Anderson, D., & Beals-Reid C. (2019). Project ECHO chronic pain: a qualitative analysis of recommendations by expert faculty and the process of knowledge translation. Journal of Continuing Education in the Health Professions, 39, 194.

  36. 36.

    London, K., Carey, M., & Russell, K. (2019). Community health worker certification requirements by state: Connecticut. Retrieved November 30, 2019 from https://www.cthealth.org/wp-content/uploads/2016/02/CHW-Certificaiton-by-State-Final-Final.pdf.

Download references

Acknowledgements

Not applicable.

Funding

No funding for the research design of the study and collection, analysis, and interpretation of data and in writing the manuscript was provided.

Author information

Affiliations

Authors

Contributions

AJD, SR, ML, and AR contributed to the study conception and design. SR and FM led the acquisition of the data. AJD and SR conducted the analysis and interpretation of the data. AJD and SR wrote the initial draft of the manuscript. AP and DA provided critical revisions of the initial draft. All authors contributed to subsequent reviews and revisions, and approved the final manuscript.

Corresponding author

Correspondence to April Joy Damian.

Ethics declarations

Ethics approval and consent to participate

The Institutional Review Board at the Community Health Center, Inc. approved all study procedures and waived the need for written and signed consent from participants.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1:.

Appendices A-D.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Damian, A.J., Robinson, S., Manzoor, F. et al. A mixed methods evaluation of the feasibility, acceptability, and impact of a pilot project ECHO for community health workers (CHWs). Pilot Feasibility Stud 6, 132 (2020). https://doi.org/10.1186/s40814-020-00678-y

Download citation

Keywords

  • Community health workers
  • Project ECHO
  • Workforce training
  • Health equity