Michigan Model for HealthTM Learning to Enhance and Adapt for Prevention (Mi-LEAP): protocol of a pilot randomized trial comparing Enhanced Replicating Effective Programs versus standard implementation to deliver an evidence-based drug use prevention curriculum
Pilot and Feasibility Studies volume 8, Article number: 204 (2022)
School-based drug use prevention programs have demonstrated notable potential to reduce the onset and escalation of drug use, including among youth at risk of poor outcomes such as those exposed to trauma. Researchers have found a robust relationship between intervention fidelity and participant (i.e., student) outcomes. Effective implementation of evidence-based interventions, such as the Michigan Model for HealthTM (MMH), is critical to achieving desired public health objectives. Yet, a persistent gap remains in what we know works and how to effectively translate these findings into routine practice. The objective of this study is to design and test a multi-component implementation strategy to tailor MMH to meet population needs (i.e., students exposed to trauma), and improve the population-context fit to enhance fidelity and effectiveness.
Using a 2-group, mixed-method randomized controlled trial design, this study will compare standard implementation versus Enhanced Replicating Effective Programs (REP) to deliver MMH. REP is a theoretically based implementation strategy that promotes evidence-based intervention (EBI) fidelity through a combination of EBI curriculum packaging, training, and as-needed technical assistance and is consistent with standard MMH implementation. Enhanced REP will tailor the intervention and training to integrate trauma-informed approaches and deploy customized implementation support (i.e., facilitation). The research will address the following specific aims: (1) design and test an implementation strategy (Enhanced REP) to deliver the MMH versus standard implementation and evaluate feasibility, acceptability, and appropriateness using mixed methods, (2) estimate the costs and cost-effectiveness of Enhanced REP to deliver MMH versus standard implementation.
This research will design and test a multi-component implementation strategy focused on enhancing the fit between the intervention and population needs while maintaining fidelity to MMH core functions. We focus on the feasibility of deploying the implementation strategy bundle and costing methods and preliminary information on cost input distributions. The substantive focus on youth at heightened risk of drug use and its consequences due to trauma exposure is significant because of the public health impact of prevention. Pilot studies of implementation strategies are underutilized and can provide vital information on designing and testing effective strategies by addressing potential design and methods uncertainties and the effects of the implementation strategy on implementation and student outcomes.
NCT04752189—registered on 8 February 2021 on ClinicalTrials.gov PRS
School-based universal prevention interventions have demonstrated notable potential to reduce the onset and escalation of drug use and mental health problems, including among youth exposed to trauma, marginalization, and socioeconomic disadvantage [1, 2]. Universal prevention interventions, also referred to as Tier 1, are delivered to an entire population regardless of risk . These interventions can have a lasting impact on youth by reducing or preventing multiple interrelated outcomes (e.g., drug use and poor mental health) that share common risk factors [4,5,6]. School-based prevention can also reach large populations of young people, including those underserved in other settings . Thus, schools are a critically important setting in which to support well-being and mitigate the effects of risk exposure among children and youth. Tier 1 prevention that is responsive to population needs offers a promising opportunity to reduce the short- and long-term consequences of exposure to stress and adversity, including substance abuse, the development of substance use disorders, mental illness, and academic failure, by enhancing resilience, providing a supportive context, and avoiding stigmatization and retraumatization [1, 8].
Recent research indicates that trauma exposure is pervasive among youth. An estimated 30.5% of youth ages 12–17 are exposed to multiple (2 or more) Adverse Childhood Experiences or ACEs . ACEs are potentially traumatic events that occur during childhood including abuse, neglect, witnessing violence, parental substance abuse, and mental health problems [9, 10]. School-based, trauma-informed interventions represent a promising way to mitigate the impact of exposure to adversity on children and youth, especially given the reach of universal prevention and the high prevalence of trauma exposure in the general population . Researchers have found higher rates of ACEs, other trauma, and toxic stress exposure among youth experiencing marginalization and socioeconomic disadvantage .
Taken together, this research indicates that school-based universal prevention would benefit from incorporating approaches to meet the needs of trauma-exposed youth. By incorporating trauma-informed approaches, teachers and other school professionals can reduce the risk of additional adversity exposure and retraumatization and strengthen factors that support resilience [12, 13]. Evidence-based interventions (EBIs), however, have rarely been designed to remain responsive to student needs, such as trauma exposure . As a result, EBIs frequently fail to achieve desired public health outcomes, including among those who would most benefit [15, 16]. Researchers suggest that the public health impact of EBIs can be improved by addressing key determinants (or barriers) and facilitators of successful implementation (see Fig. 1). this would enhance the adoption, delivery, and sustainment of EBIs, as well as bridge the sizable gap between knowing which prevention strategies work and effectively translating such strategies into routine practice [17, 18]. Designing and deploying implementation strategies for existing universal prevention EBIs, such as the Michigan Model for HealthTM (MMH), offers an efficient way to address key barriers to implementation, meet population needs, and achieve public health objectives.
MMH is a theoretically-based, universal prevention curriculum that has demonstrated efficacy in randomized trials in reducing substance use and improving mental health outcomes among high school students [21, 22]. The curriculum is grounded in social cognitive theory  and the health belief model . Our current study focuses on three core units of the MMH curriculum: the foundational skills unit, social and emotional health, and alcohol, tobacco, and other drugs. The MMH curriculum is recognized as an evidence-based intervention by CASEL (the Collaborative on Academic and Social and Emotional Learning) and is aligned with Michigan and National (USA) Health Education standards [25, 26]. The curriculum is widely adopted across Michigan, with 91% of health teachers using MMH . Yet, similar to other EBIs, MMH is infrequently delivered with fidelity . A statewide study found that 58% of educators failed to meet state-designated MMH fidelity standards  (i.e., delivering 80% or more of the curriculum); this is even higher among schools in economically challenged communities, with 73% not meeting state fidelity standards .
Replicating effective programs (REP) is a well-suited implementation strategy for school-based prevention. REP is a multi-component strategy used in community settings, including schools, focused on enhancing fit between the intervention and context while maintaining fidelity to core EBI functions. It is based on the CDC’s research-to-practice framework [30, 31] and guided by social cognitive  and diffusion of innovations theories . REP is a low-level strategy that is consistent with the standard implementation of the MMH curriculum and includes three primary components: curriculum packaging materials, teacher training, and as-needed technical assistance (Table 1). As REP is not always sufficient to effectively implement complex behavioral interventions , researchers developed Enhanced REP that includes tailoring of curriculum materials, tailored training, and ongoing provider consultation, or Facilitation. Facilitation is based on the integrated Promoting Action on Research Implementation in Health Services (iPARIHS) framework, to provide more intensive implementation support [34, 35]. Researchers have found enhanced program uptake among clinical sites deploying multi-component implementation strategies such as Enhanced REP . This additional tailoring and support can aid in mitigating barriers, enhancing intervention-context fit, and ultimately achieving desired public health outcomes . Additionally, the cost of implementation fundamentally influences program delivery in schools, which often have competing demands and carefully allocated resources [7, 37]. To date, most economic evaluation has focused on intervention costs and not the costs of implementation strategies required to deploy and sustain them . Systematic examination of costs and outcomes for multi-component implementation strategies is vital for scale-up and sustainability in community settings [37, 38].
Previous research identifying key determinants of the MMH curriculum implementation established that teachers found the intervention was unable to consistently meet students’ needs (i.e., the context), in particular among students experiencing marginalization, trauma, and disadvantage, posed challenges to intervention acceptability, which, in turn, reduced fidelity . While the teachers reported the curriculum is adaptable, they also reported that more intensive adaptations, including those designed to meet the needs of specific subgroups of students, were time and resource-intensive . Teachers reported these adaptations were, however, critical and one teacher reported that “(the curriculum) turn(ed) students away, and it turned the group of students away who needed it most. The students who have substance abuse problems, have emotional/mental health problems .” Collectively, this research underscores the need for implementation strategies to facilitate the responsiveness of EBIs for those disproportionately at risk of marginalization, trauma, and ultimately substance misuse, abuse, and the development of substance use disorders [42, 43]. Deploying implementation strategies that incorporate systematic adaptations to improve intervention-context fit and needed training and support is a promising approach to enhancing intervention acceptability, and fidelity to ultimately achieving improved health outcomes. An important first step, however, is executing a pilot study designing and testing the feasibility of the implementation strategies; pilot studies of implementation strategies are underutilized and can provide vital information on designing and testing effective strategies through addressing potential design and methods uncertainties and assessing potential effects of the implementation strategy on implementation and student outcomes .
The goal of the study is to design and test a multi-component implementation strategy (Enhanced REP) to enhance the effective delivery of a comprehensive health curriculum in community schools serving youth exposed to trauma and compare it to standard implementation.
The primary aim of this study will be to design and test Enhanced REP to deliver MMH. We will evaluate the feasibility, acceptability, and appropriateness of the implementation strategy, compared to standard REP (i.e., standard implementation) which will be the basis of a larger hybrid type 3 cluster-randomized trial.
The second primary aim is to conduct an economic evaluation to estimate implementation strategy costs and cost-effectiveness of Enhanced REP versus standard implementation to deliver MMH in preparation for a larger trial.
The secondary aims are to evaluate the potential effectiveness of Enhanced REP versus standard implementation on youth outcomes including student drug use, drug use risk perceptions, quality-adjusted life years, and student-level fidelity (e.g., satisfaction, curriculum engagement). We will also assess fidelity to the MMH curriculum using a teacher dose delivered measure, consistent with previous MMH implementation research [27, 29, 45].
Sample and setting
This pilot study will include ten schools from two intermediate school districts (ISDs: provide general education and curriculum support to multiple school districts), or Regional Educational Service Agencies, in Michigan. In 2019, 20.5% of youth nationally and 22.0% in Michigan experienced 2 or more ACEs and the risk of exposure increased to 24.2% for youth who experienced economic hardship .
The implementation strategy will be deployed by the school health coordinators as they are key MMH implementation intermediaries. This is also consistent with current practices; school health coordinators work with schools across Michigan serving in 24 regional hubs to support health programming such as MMH . The health coordinators maintain relationships with school districts and health teachers and provide support including training, technical assistance, and consultation for school health programs, practices, and policies . Thus, by using existing infrastructure and capacity for deploying Enhanced REP to support teachers in delivering MMH with fidelity, we will enhance the likelihood of sustainment.
As a first step in designing Enhanced REP, we will convene an advisory board of stakeholders (e.g., school health coordinators, teachers, other school professionals, and admin). Given preliminary data findings and the rates of community-level trauma and drug use in the state, we will consult experts in trauma-informed approaches to support curriculum tailoring and advise on trauma-informed training options. We will work with the advisory board to identify key areas of the curriculum that can be tailored, that is, the form aspects of the intervention. We will distinguish these elements from the core elements needed to be retained to support MMH effectiveness. This process will be guided by systematic adaptation steps described by Escoffrey et al.  and the foundational theories (the health belief model and social cognitive theory) of the intervention, to ensure that we incorporate fidelity-consistent adaptations that will not compromise the curriculum effectiveness [23, 24, 49]. We will then integrate the proposed changes into the curriculum materials. This will include replacing out-of-date materials, incorporating trauma-informed mental and emotional health resources, making the format user-friendly and flexible for teachers, updating drug use information and resources, and adding new activities for students (e.g., online interactive activities). The advisory committee members will each receive $250 remuneration for participation in study activities.
The facilitation component will be based on the Quality Enhancement Research Initiative (QUERI) implementation facilitation strategy and iPARIHS framework [50, 51]. Facilitation promotes provider (i.e., teacher) capacity and self-efficacy in addressing barriers to MMH implementation . The health coordinators will receive specialized training in facilitation based on the QUERI training program and a school-based trial deploying facilitation , adapted to Tier 1 prevention. Health coordinators will support teachers in implementing the tailored curriculum by engaging in the facilitation activities (see Table 2 for sample facilitation schedule and activities).
The training component will include asynchronous and synchronous modules focused on trauma-informed approaches with teachers as the intended audience. Before conducting training, we will assess teachers’ exposure to and awareness of trauma-informed practices to tailor the training to meet their needs. Sample modules for the trauma-informed training component of the implementation strategy bundle are included in Table 3.
Taking into consideration school closures and inconsistent education delivery in the 2020-2021 school year, we delayed the beginning of the study to September of 2021. The primary reason for this was to ensure conditions among intervention and control groups were as similar as possible and to decrease the likelihood that outcomes would be confounded by the learning modality of a school such as virtual compared to in-person learning. During this time, we planned to conduct user-testing of the tailored curriculum component of Enhanced REP. Two teachers who did not qualify to participate in the trial portion of the study tested the digital content and provided feedback. Following this user testing, the study team will meet with teachers and health coordinators to discuss feedback and identify areas for refinement prior to pilot study initiation.
In cooperation with the school health coordinators, we will identify and recruit 10 high schools currently using MMH that fail to meet state standards for implementation (implementing less than 80% of the curriculum) and/or facing one or more barriers to MMH implementation, for participation. We will focus on schools where at least 20% of students are eligible for free and reduced lunch, as socioeconomic disadvantage is an additional risk factor for ACEs. We will match participating schools a priori on key characteristics such as school size and percent of students eligible for free/reduced lunch to ensure balance across study conditions . We will randomize schools to either standard implementation (akin to standard REP) or Enhanced REP (see Table 1, Fig. 2). For the standard implementation condition, teachers will receive the MMH curriculum manual, standard training, and as-needed technical assistance, provided to them by the health coordinators. The Enhanced REP condition will receive an MMH curriculum manual tailored to incorporate trauma-informed approaches, tailored trauma-informed training, and implementation facilitation.
The health coordinators and study team will meet with teachers and administrators at participating schools to share study information. We will discuss specific procedures around data collection. Teachers will provide informed consent prior to the start of the study. We will conduct semi-structured interviews with teachers pre-implementation to assess readiness and post-implementation to assess the feasibility, acceptability, and appropriateness of standard implementation and Enhanced REP. Teachers will also complete a post-implementation survey to evaluate these constructs quantitatively. Students will be eligible for student-level surveys. Students will be recruited in conjunction with school district partners. The study team will communicate with teachers and administration, either in-person or using video conferencing and via email, to share study information.
The student survey questions will be similar to other school-based surveys (i.e., Mi-PHY: Michigan Profile for Healthy Youth), and we will follow a similar procedure for survey administration. The teachers will send a letter to parents to provide information and an opportunity to opt-out prior to the start of the study. Following parental letter and participant assent, students will complete a self-administered questionnaire through a secure, online server. As MMH is integrated as part of the school curriculum, students will complete the initial survey during their health class before the MMH delivery and at the end of the term. Students who do not assent will receive an alternate activity. We expect 300 students to participate, based on an 80% response rate across the schools, similar to other youth studies . Each school will receive $500 to support their participation in the study and teachers will receive up to $300 per academic term as remuneration for study activities.
Primary aim 1
Compare deploying Enhanced REP with standard implementation to deliver the MMH and evaluate the feasibility, acceptability, and appropriateness of the implementation strategy among teachers.
Feasibility, acceptability, and appropriateness
To evaluate comprehensively feasibility, acceptability, and appropriateness we will adopt a convergent mixed methods design (see Fig. 3). The purpose of a convergent design is to obtain “different but complementary data on the same topic . We will use Weiner et al.’s  measures to assess acceptability, appropriateness, and feasibility. Each construct has four items (e.g., REP is appealing, REP seems suitable), using a 5-point Likert scale: 1—strongly disagree to 5—strongly agree. The interview guide will focus on eliciting feedback on specific Enhanced REP components (manual, training, and facilitation) and existing challenges with curriculum implementation.
Data analytic approach
We will use an inductive/deductive thematic analytical approach outlined by Hsieh and Shannon for interview transcripts . First, each member of the study team will review the transcript material to develop a broad understanding of the content . Second, the empirical material contained in the interviews will be independently coded by project team members to condense the data into analyzable units. Segments of text ranging from a phrase to several paragraphs will be assigned codes based on a priori or emergent themes (also known as open coding; ). Codes will also be assigned to describe connections within and between categories and subcategories (axial coding; ). Third, the text will be independently coded by at least two study team members. Disagreements will be resolved through consensus and the team will develop a final codebook. Using this codebook, two study team members will separately review transcripts to determine the level of agreement in the codes applied . Fourth, based on these codes, we will use qualitative software to generate a series of categories connecting text segments grouped into separate “nodes.” We will use these nodes to examine the association between different a priori and emergent categories. Fifth, the different categories will be further condensed into broad themes.
We will evaluate appropriateness, acceptability, and feasibility using descriptive analyses from teacher surveys. The analyses will focus on descriptive statistics for the quantitative data from providers, including means, standard deviations, and proportions as appropriate.
Results from each data set will be examined side-by-side to explore convergence (i.e., comparing analysis conclusions) to investigate if qualitative and quantitative results concur and complementarity (i.e., one set of findings elaborates on another). We will also investigate how interview results elaborate on quantitative results (expansion) to deepen our understanding of why and how Enhanced REP may or may not be acceptable, feasible, and appropriate and how this may influence fidelity . Should discordant findings arise, we will use a method of managing such findings as described by Creswell and Plano Clark, such as collecting additional data, re-analyze existing datasets, and identifying potential sources of bias .
Primary aim 2
Conduct an economic evaluation to estimate the costs and cost-effectiveness of deploying Enhanced REP versus standard implementation to deliver MMH.
We will monitor the activities listed in Table 4 to estimate implementation strategy costs. The study team will track time for labor costs, which will constitute most of the resources/costs. This will include time logs for health coordinators to track Enhanced REP Facilitation activities, costs for training, and indirect costs associated with teacher time to attend tailored training and participate in implementation support. The study team will also track and compile all non-labor costs (e.g., adapted curriculum website updates and maintenance).
Researchers have used micro-costing approaches frequently in implementation science . One approach is a modified cost calculator approach that has been applied in the Costs of Implementing New Strategies (COINS); this approach identifies a range of costs across phases of implementation (e.g., pre-implementation, implementation, and sustainability) tailored to the strategies utilized for a specific implementation effort and focused on the perspective of the organization/provider deciding to adopt the EBI [63,64,65,66]. This is useful in identifying costs related to implementation for several reasons: (1) it aids in determining direct costs of implementation through tallying time spent on activities in each phase of implementation strategy deployment (often the bulk of implementation strategy costs), (2) this practical approach can provide needed guidance and scaffolding for stakeholders and decision-makers to determine implementation costs so organizations could accurately estimate the necessary resources for implementation success, and (3) this approach has been used previously with Enhanced REP in estimating costs as the first step in cost-effectiveness analysis for a community-based clinical trial . Table 4 provides a list of anticipated activities whose costs will be estimated prospectively as part of the pilot trial. We will also assess the costs of the REP condition, or standard implementation of MMH, for the intervention materials and training using available data from MDHHS and the Michigan School Health Coordinators Association (MiSHCA). For the standard implementation condition, we will ask coordinators to track time spent on as-needed technical assistance.
We will use the EQ-5D to assess Quality Adjusted Life Years (QALYs). The EQ-5D is a multi-attribute utility instrument that yields interval-level scores ranging from 0 (dead) to 1 (perfect health) . This mapping provides a health utility measure for each health state experienced by patients in the study and can be used to calculate quality-adjusted life years, the preferred measure for health benefits used in cost-effectiveness analysis .
Data analytic approach
All costs will be adjusted to the current year's US dollars. Costs will include implementation strategy costs (i.e., inputs) listed in Table 4, such as labor costs for meetings, costs associated with training, and labor costs associated with the provision of facilitation. We will estimate the costs of Enhanced REP and standard implementation using the cost data, with the comparator strategy being standard implementation. Our primary utility will be a change in reported quality-adjusted life years (QALYs). We will also assess changes in student drug use during the intervention period. We will use net costs (net increase in costs from Enhanced REP compared to standard implementation) and net effectiveness (net change in drug use for Enhanced REP versus standard implementation) to estimate the incremental cost-effectiveness ratio for student outcomes. We will conduct a one-way sensitivity analysis on all cost input parameters listed in Table 4 as well as health utilities to provide estimates of the costs and incremental cost-effectiveness to decision-makers. The analysis will also include multi-way sensitivity analyses on the parameters whose results are most sensitive in influencing the cost-effectiveness ratio . This will inform the feasibility of our costing approach, provide preliminary information on cost input distributions and which inputs may be especially influential on cost-effectiveness ratios to inform the detail of future data collection efforts; this pilot study will also provide preliminary information on the cost-effectiveness of Enhanced REP to inform the utility of undertaking a CEA in a larger trial .
Our secondary aim is to evaluate the potential effectiveness of Enhanced REP versus standard implementation on student outcomes. Student outcomes include drug use, drug use risk perceptions, and student-level fidelity (e.g., satisfaction, curriculum engagement) as described by Barrera et al. . We will also assess MMH fidelity using the teacher dose delivered.
Secondary aim study measures are summarized in Table 5.
We will assess past 3- and 12-month substance use using items from Monitoring the Future (MTF; ) with adapted response options and timeframe.
Implementation outcomes, student-level
Engagement has been identified as a fidelity dimension that provides information on participant responsiveness to the intervention and is key to intervention success . As described in Barrera et al. , we will assess engagement using student satisfaction and key intervention skills. The satisfaction measure will be adapted based on a scale developed by Giles et al. for another drug prevention intervention with good psychometric properties that will include four items . We will evaluate key intervention skills: assertive communication, refusal skills, and decision-making. These dimensions are identified in the MMH curriculum summative evaluation materials, assessed in previous MMH studies, and based on National Health Education Standards [21, 26]. Students will rate their level of agreement using a 5-point Likert scale on their proficiency with specific elements related to each skill.
Fidelity, teacher-level: MMH dose delivered
We will assess the dose or amount of the intervention delivered using a curriculum fidelity tracking form. Teachers will complete a brief form following each lesson and unit included in the study. Lessons are grouped into multiple units, including the alcohol, tobacco, and other drug prevention unit, the skills unit, and the social and emotional learning unit. We will assess the dose delivered by calculating the total number of lessons completed within each unit (10 lessons/unit). We will also ask teachers to report any adaptations or modifications, guided by the framework proposed by Wiltsey-Stirman et al.  on the tracking form. Modifications include adding, removing, and changing content, substituting activities, and changing activity formats.
The focus of this pilot trial will be to generate information to inform our approach to collecting student-level data and estimate the treatment effects variance estimates for a larger study . We will calculate and examine treatment effects using linear mixed-effects models (LMM), with an understanding of potential challenges with estimating treatment effects for pilot studies . LMM are appropriate models for analyzing clustered data and may involve fixed and random effects . We will control for key demographic variables in all analyses, including race/ethnicity gender, and socioeconomic status.
We will assess MMH dose delivered using the sum score across all units as well as unit-specific dose delivered calculations. Consistent with previous research, we will also calculate the proportion of the units delivered, individually and combined, with a focus on descriptive statistics (means, standard deviations).
This research has the potential to support the design and deployment of effective and sustainable strategies for implementing drug use prevention interventions in schools. Preventing or tempering the onset and escalation of drug use can reduce the burden of social, emotional, and economic costs placed on youth and their families, communities, and society . The proposed research can contribute to improving public health and reducing health disparities related to the disparate consequences of drug use and addiction among youth exposed to trauma. Systematically designing and testing implementation strategies to address key determinants implementation, including poor fit between the intervention and context/population needs, will support refinement and ultimately, leadership engagement in ongoing implementation support once the research study ends. Economic evaluation can provide vital information for stakeholders and decision-makers regarding implementation strategies to support and sustain quality EBI delivery.
This project has several strengths. First, we will design and systematically deploy a theoretically grounded bundle of implementation strategies, Enhanced REP, to address key determinants. Second, researchers have noted that many full implementation trials have progressed without initial pilot trials ; thus, this research will support the quality and rigor of full implementation trials by providing an opportunity to refine the implementation strategy and address issues around the feasibility of research methods to enhance the contributions to the field . This research has the potential to support the advancement of research-to-practice translation of substance use prevention programs through designing implementation strategies that are effective, sustainable, and that improve quality program delivery. Third, we will use a community-engaged approach to designing the implementation strategy by incorporating an advisory board composed of multiple stakeholders. Fourth, we will estimate the costs and cost-effectiveness of the implementation strategies versus standard implementation to help address a central challenge to the effective implementation and sustainment of EBIs. This has the potential to provide useful, accessible information for communities to make well-informed decisions about resource allocation and implementation. Fourth, this project has the potential to advance implementation strategies for universal substance use prevention interventions. Researchers have estimated that effectively implementing school-based prevention would save an estimated lifetime monetary cost of $33.5 billion and a total cost of $98.6 billion to society for early adolescents using costs estimated in 2002 . Finally, this research will support using implementation science to meet the needs of underserved families and communities who are at increased risk of trauma exposure, drug use, and its consequences.
We also note several potential study limitations. While the study will work with schools serving under-resourced families across two counties in Michigan, schools are heterogeneous settings with different organizational structures. Thus, different schools may experience other relevant barriers at various levels not addressed in the Enhanced REP strategy bundle. Alternative strategies may be needed to support successful implementation, including strategies in the outer setting, and may be an important focus of future research. We did not include students explicitly in the implementation strategy design process. Although implementation strategies often focus on the intervention deliverer and relevant contextual considerations, incorporating the input of the intervention recipients in addition to assessing their responsiveness (e.g., engagement) is an important future direction and potential focus of a larger trial.
The next step in this program of research would be to test the implementation strategy on a larger scale over a longer duration, including a large-scale cluster-randomized trial Type 3 hybrid trial. This would be conducted in schools serving youth experiencing ACEs and other forms of adversity across multiple states adopting the MMH curriculum. This research will also support scaling up these strategies to improve health curriculum delivery and advance substance use prevention.
Availability of data and materials
Electronic copies of publications will be made accessible to a journal in PubMed Central. This article is licensed under the Creative Commons Attribution 4.0 Generic License (CC BY 4.0), which permits use, sharing, adaptation, distribution, and reproduction if appropriate credit to the original author(s) is provided along with a link to the Creative Commons license and an indication of changes. De-identified primary participant (i.e., student)-level data will be available through an appropriate data repository, such as the NIH HEAL (National Institutes of Health: Helping to End Addiction Long-termSM) Initiative central data repository. This data will be available upon acceptance for publication of the main findings from the final student-level dataset. Data will be available in the NIH HEAL repository per HEAL guidelines. Access to individual-level data will require entering into a data-sharing agreement that includes requirements to protect participants’ privacy and data confidentiality.
Adverse childhood experiences
Collaborative for Academic, Social, and Emotional Learning
Centers for Disease Control and Prevention
Costs of Implementing New Strategies
Data Safety and Monitoring Board
- Enhanced REP:
Enhanced Replicating Effective Programs
Health Insurance Portability and Accountability Act
Integrated Promoting Action on Research Implementation in Health Services
Institutional Review Board
Intermediate School Districts
Linear mixed-effects models
Michigan Model for HealthTM: Learning to Enhance and Adapt for Prevention
Michigan Model of HealthTM
Monitoring the Future
National Institute on Drug Abuse
- NIH HEAL:
National Institutes of Health Helping to End Addiction Long-termSM
National Survey on Drug Use and Health
Quality-adjusted life years
Quality Enhancement Research Initiative
Substance use disorders
Herrenkohl T, Hong S, Verbrugge B. Trauma-informed programs based in schools: linking concepts to practices and assessing the evidence. Am J Community Psychol. 2019;64(3–4):373–88.
Temkin D, Harper K, Stratford B, Sacks V, Rodriguez Y, Bartlett JD. Moving policy toward a whole school, whole community, whole child approach to support children who have experienced trauma. J Sch Health. 2020;90(12):940–7.
O’Connell M, Boat T, Warner K, Warner K. Preventing mental, emotional and behavioral disorders among young people: progress and possibilities. Washington, DC: Board on Children, Youth, and Families, Division of Behavioral and Social Sciences and Education, National Academies Press; 2009. p. 549.
Greenberg M. School-based prevention: current status and future challenges. Effect Educ. 2010;2(1):27–52.
Hale D, Fitzgerald-Yau N, Viner R. A systematic review of effective interventions for reducing multiple health risk behaviors in adolescence. Am J Public Health. 2014;104(5):e19–41.
Hill KG, Bailey JA, Steeger CM, Hawkins JD, Catalano RF, Kosterman R, et al. Outcomes of childhood preventive intervention across 2 generations: a nonrandomized controlled trial. JAMA Pediatr. 2020;174(8):764–71.
Lee R, Gortmaker S. Health dissemination and implementation within schools. In: Brownson R, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science to practice. 2nd ed. New York: Oxford University Press; 2018. p. 401–16.
SAMHSA. Trauma-informed approach and trauma-specific interventions: Substance Abuse and Mental Health Services Administration (SAMHSA); 2014. Available from: https://www.samhsa.gov/nctic/trauma-interventions. Cited 2019 Feb 4
Bethell CD, Newacheck P, Hawes E, Halfon N. Adverse childhood experiences: assessing the impact on health and school engagement and the mitigating role of resilience. Health Aff. 2014;33(12):2106–15.
CDC. Preventing Adverse Childhood Experiences (ACES): leveraging the best available evidence. Atlanta: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention; 2019.
Klevens J, Metzler M. Bringing a health equity perspective to the prevention of child abuse and neglect. In: Lonne B, Higgins D, Scott D, Herrenkohl, editors. Re-visioning public health approaches for protecting children; 2019. p. 330–59.
Chafouleas SM, Johnson AH, Overstreet S, Santos NM. Toward a Blueprint for Trauma-Informed Service Delivery in Schools. School Mental Health. 2016;8(1):144–62.
Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB. The impact of enhancing students’ social and emotional learning: a meta-analysis of school-based universal interventions. Child Dev. 2011;82(1):405–32.
Castro FG, Yasui M. Advances in EBI Development for Diverse Populations: Towards a Science of Intervention Adaptation. Prev Sci. 2017;18(6):623–9.
Durlak J, DuPre E. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3):327.
U.S. Department of Education. Prevalence and implementation fidelity of research-based prevention programs in public schools: final report. Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Programs Study Service; 2011. Report no.: ED-00-CO-0119
Grol R, Grimshaw J. Evidence-based implementation of evidence-based medicine. Jt Comm J Qual Improv. 1999;25(10):503–13.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation Science. 2013;8(1).
Proctor E, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research. 2009;36(1):24–34.
Lyon A, Bruns E. From Evidence to Impact: Joining Our Best School Mental Health Practices with Our Best Implementation Strategies. School Mental Health. 2019;11(1):106–14.
O’Neill J, Clark J, Jones J. Promoting mental health and preventing substance abuse and violence in elementary students: a randomized control study of the michigan model for health. J Sch Health. 2011;81(6):320–30.
Shope J, Copeland L, Maharg R, Dielman T. Effectiveness of a high school alcohol misuse prevention program. Alcohol Clin Exp Res. 1996;20(5):791–8.
Bandura A. Human agency in social cognitive theory. Am Psychol. 1989;44(9):1175–84.
Rosenstock IM. Historical origins of the health belief model. Health Educ Monogr. 1974;2(4):328–35.
CASEL. Program Guide CASEL: Collaborative for Academic Social and Emotional Learning. [Internet]. Program Guide. 2022. Available from: https://pg.casel.org/. Cited 8 Feb 2022.
CDC. National Health Education Standards [Internet]. CDC Healthy Schools. 2019. Available from: https://www.cdc.gov/healthyschools/sher/standards/index.htm. Cited 21 Dec 2021.
Rockhill S. Use of the Michigan Model for Health Curriculum among Michigan Public Schools. Lansing MI; 2017.
Durlak J, Weissberg R, Pachan M. A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. Am J Community Psychol. 2010;45(3–4):294–309.
Eisman A, Kilbourne A, Ngo Q, Fridline K, Zimmerman M, Greene D, et al. Implementing a state-adopted high school health curriculum: a case study. J Sch Health. 2020;90(6):447–56.
Neumann M, Sogolow E. Replicating effective programs: HIV/AIDS prevention technology transfer. AIDS education and prevention: official publication of the International Society for AIDS Education. 2000;12(5 Suppl):35–48.
Kilbourne AM, Smith SN, Choi SY, Koschmann E, Liebrecht C, Rusch A, et al. Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools. Implementation Sci. 2018;13(1):119.
Bandura A. Social learning theory: Prentice-Hall; 1977. Available from: http://mirlyn.lib.umich.edu/Record/000313256 CN - LB1084 .B3571
Rogers E. Diffusion of innovations. 2003. Available from: http://mirlyn.lib.umich.edu/Record/004335364 CN - HM 101 .R72 2003
Kilbourne AM, Almirall D, Eisenburg D, Waxmonsky J, Goodrich DE, Fortney JC, et al. Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implement Sci. 2014;9(1):132.
Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2015;11(1):33.
Kilbourne AM, Almirall D, Goodrich DE, et al. Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy. Implement Sci. 2014;9:163. Published 2014 Dec 28. https://doi.org/10.1186/s13012-014-0163-3.
Raghavan R. The role of economic evaluation in dissemination and implementation research. In: Dissemination and implementation research in health: translating science to practice. 2nd ed. New York: Oxford University Press; 2018. Available from: http://mirlyn.lib.umich.edu/Record/011515836 CN.
Eisman A, Kilbourne A, Dopp A, Saldana L, Eisenberg D. Economic evaluation in implementation science: making the business case for implementation strategies. Psychiatry Res. 2020;283:112433.
Kilbourne A, Goodrich D, Nord K, Van Poppelen C, Kyle J, Bauer M, et al. Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices. Administration and policy in mental health. 2015;42(5):642–53.
Eisman A, Kiperman S, Rupp L, Kilbourne A, Palinkas L. Understanding Key Implementation Determinants for a School-Based Universal Prevention Intervention: A qualitative study. Translational Behavioral Medicine. 2022;12(3):411–22.
Eisman A, Palinkas L, Herrenkohl T, Boyd C, Kilbourne A. Meeting the needs of trauma-exposed youth: enhancing implementation fidelity to a state-wide health curriculum. Washington DC: Academy Health and NIH; 2020.
Baumann A, Cabassa L. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res. 2020;20(190):9.
LeTendre M, Reed M. The effect of adverse childhood experience on clinical diagnosis of a substance use disorder: results of a nationally representative study. Subst Use Misuse. 2017;52(6):689–97.
Pearson N, Naylor P, Ashe M, Fernandez M, Yoong S, Wolfenden L. Guidance for conducting feasibility and pilot studies for implementation trials. Pilot Feasibility Stud. 2020;6(1):167.
Eisman A, Kilbourne A, Greene D, Walton M, Cunningham R. The user-program interaction: how teacher experience shapes the relationship between intervention packaging and fidelity to a state-adopted health curriculum. Prev Sci. 2020;21(6):820–9.
United Health Foundation. Adverse childhood experiences: America’s health rankings analysis of U.S. HHS, HRSA, Maternal and Child Health Bureau (MCHB), Child and Adolescent Health Measurement Initiative (CAHMI), National Survey of Children’s Health Indicator Data Set, Data Resource Center for Child and Adolescent Health. 2020. Available from: https://www.americashealthrankings.org/explore/health-of-women-and-children/measure/ACEs/state/MI. Cited 2020 Jul 10.
MiSHCA. MiSHCA. Available from: https://mishca.org/. Cited 2021 Feb 25.
Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, et al. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci. 2018;13(1):125.
Wiltsey Stirman S, Baumann A, Miller C. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1):58.
Kirchner J, Waltz T, Powell B, Smith J, Proctor E. Implementation strategies. In: Brownson R, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science to practice. 2nd ed. New York: Oxford University Press; 2018.
Ritchie M, Dollar K, Miller C, Oliver K, Smith J, Lindsay J, et al. Using implementation facilitation to improve care in the Veterans Health Administration (version 2): Veterans Health Administration, Quality Enhancement Research Initiative (QUERI) for Team-Based Behavioral Health; 2017.
Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine: Oxford University Press; 2016. p. 537.
Drummond M, Coyle D. The role of pilot studies in the economic evaluation of health technologies. Int J Technol Assess Health Care. 1998;14(3):405–18.
Crespi CM. Improved designs for cluster randomized trials. Annu Rev Public Health. 2016;37(1):1–16.
Brener N, Kann L, Shanklin S, Kinchen S, Eaton D, Hawkins J, et al. Methodology of the youth risk behavior surveillance system--2013. MMWR Recomm Rep. 2013;62(RR-1):1–20.
Morse JMR. Approaches to qualitative-quantitative methodological triangulation. Nurs Res. 1991;40(2):120–3.
Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.
Creswell J, Plano Clark V. Designing and conducting mixed methods research. 3rd ed. Los Angeles: SAGE; 2018. p. 492.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
Miles MB, Huberman AM, Saldaña J. Qualitative data analysis: a methods sourcebook. 3rd ed. Thousand Oaks: SAGE Publications, Inc; 2014. p. 381.
Corbin JM, Strauss AL, Strauss AL. Basics of qualitative research: techniques and procedures for developing grounded theory. 1st ed; 1998. p. 312.
Boyatzis RE. Transforming qualitative information: thematic analysis and code development. Thousand Oaks: Sage Publications; 1998. p. 184.
Saldana L. The stages of implementation completion for evidence-based practice: protocol for a mixed methods study. Implement Sci. 2014;9(1):43.
Bowser D, Henry B, McCollister K. Cost analysis in implementation studies of evidence-based practices for mental health and substance use disorders: a systematic review. Implement Sci. 2021;16(26):15.
Holmes L, Landsverk J, Ward H, Rolls-Reutz J, Saldana K, Wulczyn F, et al. Cost calculator methods for estimating casework time in child welfare services: a promising approach for use in implementation of evidence-based practices and other service innovations. Child Youth Serv Rev. 2014;39:169–76.
Chamberlain P, Snowden L, Padgett C, Saldana L, Roles J, Holmes L, et al. A strategy for assessing costs of implementing new practices in the child welfare system: adapting the english cost calculator in the United States. Adm Policy Ment Health. 2011;38(1):24–31.
Eisman A, Hutton D, Prosser L, Smith S, Kilbourne A. Cost-effectiveness of the Adaptive Implementation of Effective Programs Trial (ADEPT): approaches to adopting implementation strategies. Implement Sci. 2020;15(1):1–13.
EuroQol Research Foundation. EQ-5D-5L User Guide, 2019. Available from: https://euroqol.org/publications/user-guide.
Barrera M, Berkel C, Castro FG. Directions for the advancement of culturally adapted preventive interventions: local adaptations, engagement, and sustainability. Prev Sci. 2017;18(6):640–8.
Johnston LD, Miech RA, O’Malley PM, Bachman JG, Schulenberg JE, Patrick ME. Monitoring the future national survey results on drug use: 1975-2017. Ann Arbor: Institute for Social Research; 2018. p. 126.
2018 National Survey on Drug Use and Health (NSDUH): CAI Specifications for Programming. Rockville, MD: Center for Behavioral Health Statistics and Quality; 2017.
Giles SM, Harrington NG, Fearnow-Kenney M. Evaluation of the All Stars Program: Student and Teacher Factors That Influence Mediators of Substance Use. J Drug Educ. 2001;31(4):385–97.
Home | Michigan Model for Health [Internet]. Available from: https://www.michiganmodelforhealth.org/. Cited 14 Dec 2020.
Stirman S, Miller C, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8(1):65.
Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10(1):10.
Sim J. Should treatment effects be estimated in pilot and feasibility studies? Pilot Feasibility Stud. 2019;5(1):1–7.
West BT, Welch KB, Galecki AT. Linear Mixed Models: A Practical Guide Using Statistical Software. 2nd ed. Chapman and Hall/CRC; 2014. [cited 2020 Feb 28].
Miller T, Hendrie D. Substance abuse prevention dollars and cents: a cost-benefit analysis. Rockville: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration; 2008.
We would like to acknowledge David Hutton, Lisa Prosser, Dana Greene, Jr., Carol Boyd, Christina Harvey, and Kelly Walsh for their contributions to the project.
This research is supported by the National Institute on Drug Abuse, K01DA044279, PI: Eisman.
Ethics approval and consent to participate
This study is recruiting high school teachers from two regions in Michigan. Before participation, teachers will be randomized into two groups: REP and Enhanced REP. The current study was approved Expedited Review Category 7 by the Wayne State University Institutional Review Board, IRB-20-10-2821. The study was registered on ClinicalTrials.gov, NCT04752189, https://clinicaltrials.gov/ct2/show/NCT04752189, February 12, 2021. We obtain teacher consent prior to participation in the survey and semi-structured interviews using an online consent form. Parental opt-out consent and student assent will be obtained before student participation in the survey. Electronic data files will be password-protected and stored on a secure server through the institution with restricted access. Consent forms will be stored separately because they contain identifying information. Qualitative data will be transcribed using a HIPAA-compliant transcription service. The proposed research will include a data safety and monitoring board (DSMB) comprised of individuals who are not affiliated with the project and have expertise and experience with data safety and monitoring and human subjects research.
Consent for publication
The authors declare they have no competing interests to report.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Eisman, A.B., Palinkas, L.A., Koffkey, C. et al. Michigan Model for HealthTM Learning to Enhance and Adapt for Prevention (Mi-LEAP): protocol of a pilot randomized trial comparing Enhanced Replicating Effective Programs versus standard implementation to deliver an evidence-based drug use prevention curriculum. Pilot Feasibility Stud 8, 204 (2022). https://doi.org/10.1186/s40814-022-01145-6