Skip to main content
Download PDF

Adaption and pilot implementation of an autism executive functioning intervention in children’s mental health services: a mixed-methods study protocol

Pilot and Feasibility Studies volume 6, Article number: 55 (2020) Cite this article

Abstract

Background

Youth with autism spectrum disorder (ASD) represent a growing population with significant service needs. Prominent among these needs are high rates of co-occurring psychiatric conditions that contribute to increased functional impairments and often necessitate mental health services. Executive functioning deficits are associated with ASD as well as common co-occurring conditions (e.g., attention-deficit/hyperactivity disorder) and an evidence-based intervention has been developed and tested to address executive functioning within the school context. There is an urgent need to implement indicated evidence-based interventions for youth with ASD receiving care in community mental health settings. Interventions that optimally “fit” the mental health services context as well as the complex and co-occurring mental health needs of these youth have the potential to improve key clinical outcomes for this high priority population.

Methods

This mixed-methods developmental study will apply the Exploration, Preparation, Implementation, Sustainment implementation framework and a community-academic partnership approach to systematically adapt and test an evidence-based executive functioning intervention for youth with ASD for delivery in community mental health settings. Specific aims are to (1) conduct a need and context assessment to inform the systematic adaptation an executive functioning evidence-based intervention; (2) systematically adapt the clinical intervention and develop a corresponding implementation plan, together entitled “Executive Functioning for Enhancing Community-based Treatment for ASD,” (EFFECT for ASD); and (3) conduct a feasibility pilot test of EFFECT for ASD in community mental health settings.

Discussion

Tailoring evidence-based interventions for delivery in community-based mental health services for youth with ASD has the potential to increase quality of care and improve child outcomes. Results from the current study will serve as the foundation for large-scale hybrid implementation and effectiveness trials and a generalizable approach for different service systems of care and clinical populations.

Trial registration

Clinicaltrials.gov, NCT04295512.

Peer Review reports

Background

Autism spectrum disorder (ASD) is characterized by impairments in socialization, communication, and restricted interests and repetitive behaviors [1]. Currently estimated to affect 1 in 59 school-aged youth, individuals with ASD represent an expanding population with multiple health care and service needs [2]. Caring for autism is costly, as ASD is the youth health care condition with the highest increase in annual expenditures—projected to increase from $268 to $461 billion by 2025 [3, 4]. Youth with ASD have high rates of co-occurring psychiatric conditions (e.g., disruptive or challenging behaviors, attention-deficit/hyperactivity disorder, anxiety), estimated at 70% versus 25% for other youth [5,6,7], that can contribute to further functional impairments and health care needs [8,9,10]. Several evidence-based interventions (EBIs) have been developed, including those targeting co-occurring psychiatric symptoms. However, the majority of youth with ASD treated in the community do not have access to these EBIs. Therefore, to maximize care effectiveness and reduce long-term costs, it is critical that youth with ASD have access to indicated EBIs via their effective implementation and sustainment in routine care settings.

Mental health services for youth with ASD

The public mental health service system plays a key role in serving school-age youth with ASD [11, 12]. Mental health providers report that youth with ASD or suspected ASD represent 21% of their caseloads [13]. Yet, the complex clinical presentations and unique challenges associated with treating youth with ASD pose significant service challenges for mental health providers. Mental health providers report frustration serving this population due to perceived limited progress and ineffective treatment strategies, and cite a need and motivation for specialized training in effective ASD interventions [13]. Caregivers of youth with ASD also report frustration and decreased satisfaction with mental health services due to providers’ limited ASD knowledge and training [14, 15]. Given the significant mental health needs of youth with ASD, targeted efforts are necessary to integrate EBIs in ways that are feasible, acceptable, and effective in regard to training providers and meeting the needs of youth with complex mental health problems.

Existing EBIs for ASD

Despite significant efforts to develop and test EBIs targeting core ASD symptoms as well symptoms associated with co-occurring psychiatric conditions, community implementation deficits persist [16,17,18]. Many EBIs demonstrate poor fit with end user (i.e., therapist and client) and service settings needs, resulting in low acceptability or feasibility and ultimately contributing to limited implementation [19, 20]. For example, most mental health EBIs for ASD target specific co-occurring conditions (e.g., cognitive behavioral therapy for ASD and anxiety [21, 22]). EBIs targeting individual co-occurring disorders may pose implementation challenges, including poor fit with client clinical needs, as children with ASD presenting to community mental health services typically present with multiple co-occurring conditions [23, 24]. These mental health disorder-specific EBIs also pose a poor fit with provider training needs. Community providers face significant challenges in sufficiently mastering and delivering an EBI or, in the case of co-occurring conditions, appropriately selecting a sequence of EBIs for implementation [18, 25]. Targeting underlying clinical mechanisms that cut across co-occurring conditions and ASD represents a potentially more impactful and feasible approach to promoting community implementation through better meeting the complex clinical needs of youth served in these settings. Moreover, the increase applicability of these EBIs may also facilitate improved implementation outcomes through addressing key determinants of EBI use such as provider attitudes and perceptions of fit [26,27,28].

ASD EBIs in mental health services

Until recently, most of the ASD intervention research was primarily aimed at examining the efficacy of interventions conducted in research settings or the effectiveness of delivery in non-mental health settings (e.g., early intervention, schools [21, 22, 29, 30]). Thus, there is a need to develop, test, and implement feasible ASD interventions for delivery in publicly funded mental health settings. In response to this need, Brookman-Frazee and colleagues [31] utilized a community-partnered approach to develop an intervention protocol entitled “An Individualized Mental Health Intervention for ASD” (“AIM HI”) consisting of a package of evidence-based strategies aimed to reduce challenging behaviors in youth with ASD and a corresponding therapist training protocol. AIM HI strategically targets the primary presenting problem (e.g., challenging behaviors) for youth with ASD in mental health services rather than the individual co-occurring diagnoses to more feasibly meet child clinical needs in community mental health settings. In collaboration with autism experts and community stakeholders, AIM HI was developed based on a systematic mixed-methods needs assessment of both child clinical needs and provider training needs [13, 15]. The iterative, community partnered and transdiagnostic (i.e., appropriate for a range of co-occurring psychiatric conditions) approach to develop and further refine AIM HI has been shown to increase therapist delivery of evidence-based strategies and improve child clinical outcomes [32,33,34,35]. Furthermore, AIM HI research highlights the relevance and utility of a transdiagnostic approach to improve clinical outcomes for a clinically challenging population of children with ASD receiving mental health services [36, 37]. The current study extends AIM HI research by applying the community partnered, context-based approach to target a new transdiagnostic clinical mechanism. Specifically, the current study will systematically adapt an existing EBI targeting executive functioning to optimize its fit with the mental health service system, primary end users of mental health providers, and end targets of youth with ASD. Additionally, through a corresponding implementation plan, the current project also targets implementation mechanisms of change, namely provider attitudes, perceptions of fit and intention to use, known to predict EBI use [26,27,28].

Executive functioning: a potent transdiagnostic clinical mechanism

Identified as a key National Institute of Mental Health Research Domain Criteria subconstruct [38], executive functioning, or “cognitive control,” is defined as a collection of self-regulatory processes such as inhibition, planning, organizing, and flexibility necessary for goal-directed behavior [39]. Mounting evidence suggests an integral role of executive functioning in both ASD and psychiatric conditions that frequently co-occur with ASD [40,41,42,43,44,45,46,47]. Executive functioning deficits increase over the course of development and are linked and contribute to increased psychiatric symptoms in youth with ASD [45, 48,49,50,51]. Given that almost 80% of youth with ASD served in mental health services have attention-deficit/hyperactivity disorder and that executive functioning deficits underlie both attention-deficit/hyperactivity disorder and ASD, executive functioning serves as a key transdiagnostic clinical mechanism impacting psychiatric conditions in youth with ASD. Thus, interventions targeting executive functioning are highly relevant for this service setting and population. Additionally, socioeconomic status is a strong predictor of executive functioning, with youth from low-income or economically disadvantaged homes demonstrating more executive deficits [52, 53]. As youth from low-income families, including those with ASD, comprise a large portion of those served in community-based mental health clinics [54], executive deficits are extremely relevant to mental health services.

Executive functioning as a target of EBIs

The role of executive functioning in the etiology of mental health conditions has spurred increased focus on this key construct in intervention development efforts [55,56,57]. A cognitive-behavioral executive functioning intervention specific to ASD, entitled Unstuck and On Target (UOT [58]), was developed for use in community school service settings with students with ASD. UOT targets impairments in flexibility, goal setting and planning, and problem solving [40, 45, 48, 59]. UOT is effective in improving flexibility, organization, and problem solving for youth with ASD as well as youth with attention-deficit/hyperactivity disorder when delivered in school settings [30, 60], supporting the classification of UOT as an EBI with good quality evidence [61]. Teachers also demonstrated high rates of fidelity, intervention satisfaction, and participation and treatment completion, supporting the acceptability and feasibility of UOT in community settings [30, 62, 63]. Further, trials testing UOT revealed no important moderators of effect in a very diverse sample (not race, language spoken in the home, income, IQ) [60, 64].

Results from UOT trials suggest that EBIs addressing mechanisms such as executive functioning have the potential to effectively address symptoms related to ASD as well as the range of co-occurring mental health symptoms and conditions. Additionally, findings demonstrate sustained improvements in problem solving a year after completion of UOT, irrespective of youth race or ethnicity and income [60]. Together, these data indicate the potential utility of UOT in mental health service settings. Through its high rates of feasibility and acceptability, UOT could help meet mental health provider training needs related to effective, targeted interventions for ASD [13]. Although UOT was designed and tested in school settings to be delivered by school staff, the intervention target and associated training model is potentially transferrable to mental health settings. UOT was designed as lesson plans specifically to be implemented in school settings and delivered in group settings by school staff. Thus, adaptation or augmentation to maximize its fit with client clinical and provider training needs and thus implementation within mental health services is needed. However, there have been no studies testing the effectiveness and utility of an executive functioning EBI in mental health settings for youth with ASD and co-occurring mental health conditions, especially one specifically developed for this service setting and high priority population.

Current project

Executive Functioning for Enhancing Community-based Treatment for ASD (EFFECT for ASD) aims to examine the utility of UOT to improve key clinical and implementation outcomes for youth with ASD when systematically adapted for delivery in public mental health services. The Exploration, Preparation, Implementation, Sustainment [65] (EPIS) implementation framework will be applied to inform the systematic adaptation of UOT and develop a corresponding implementation plan to maximize its adoption and effectiveness in community mental health service settings. Specifically developed for public service sectors, EPIS specifies the implementation process, divided into four phases (exploration, preparation, implementation, sustainment), and implementation factors (i.e., innovation factors, bridging factors) associated with two contextual levels (inner and outer contexts). The outer context details external factors such as service environment and client population characteristics whereas the inner context refers to intra-organizational factors organizational and provider characteristics. Innovation factors detail those context-specific elements key to fit between an EBI and the context in which it is implemented. Bridging factors refers to those that span and link the inner and outer contexts. For example, system level policies, funding, partnership and collaboration, and advocacy can serve to link inner and outer contexts. Given its alignment with the focus and aims of the current project, the EPIS framework will be applied to all study activities [65, 66]. As the outer context is relatively stable, the primary focus will be on the impact of relevant inner context factors, namely provider characteristics, across the first three implementation phases (i.e., exploration, preparation, and implementation). As one of the outer/inner context “bridging factors” specified in EPIS and applied in previous ASD-focused research in mental health services [67], a community-academic partnership will also be integrated to tailor EFFECT for ASD fit and impact in mental health services settings. The proposed study has three specific aims that correspond to the first three EPIS phases:

  • Aim 1. Conduct a need and context assessment to inform the systematic adaptation of UOT for implementation in youth mental health services.

  • Aim 2. Systematically adapt UOT and develop a corresponding implementation plan entitled EFFECT for ASD

  • Aim 3. Conduct a feasibility pilot test of EFFECT for ASD in community mental health settings.

Targeted implementation and clinical change mechanisms

EFFECT for ASD will target two implementation change mechanisms affecting both implementation and clinical outcomes in mental health: (1) provider attitudes and perceptions of intervention fit and (2) provider’s intention to use (see Fig. 1). Provider attitudes towards EBIs are widely linked to specific practice behaviors and predict EBI use as well as increased fidelity following training [26,27,28, 65, 68]. Changes in attitudes are also known to facilitate EBI adoption and intention to use [69, 70]. Provider perceptions of innovation fit is also a critical determinant of EBI adoption and implementation in public mental health settings [71,72,73]. Through its transdiagnostic approach (i.e., appropriate for multiple co-occurring mental health conditions) and context-specific adaptation to maximize fit, EFFECT for ASD aims to improve provider attitudes and perceptions of EBI fit and subsequent intent and actual use through the creation of one intervention that providers perceive as being feasible, acceptable, and fitting within their practice. It also addresses their reported need for training and effective interventions [13] by allowing them to effectively serve the spectrum of co-occurring psychiatric conditions seen in youth with ASD on their caseloads. This project also targets a clinical change mechanism of improved executive functioning, a potent factor impacting both ASD and psychiatric symptoms, that can lead to significant youth improvements (e.g., reduced symptoms, reduced challenging behaviors) and care effectiveness.

Fig. 1
figure 1

EFFECT for ASD intervention, mechanisms, and outcomes (adapted from the EPIS framework)

Full size image

Methods

EFFECT community-academic partnership

A community-academic partnership (CAP), defined as a partnership involving community members and academic researchers targeting a cause relevant to the community of interest [74], will be incorporated into all project aims to maximize the maximize EFFECT for ASD fit and impact in community mental health settings. In the current study, we apply a CAP model utilized in previous ASD and mental health services research [75,76,77]. The CAP will be comprised of key researchers and community stakeholders representing various levels of experience and roles serving youth with ASD, including mental health organization or agency leaders, mental health providers, caregivers of youth with ASD, and the principal investigator. Across all aims, the CAP will meet regularly to provide multiple perspectives and inform and guide research activities, including supporting participant recruitment, adaptation and development of EFFECT of ASD, and interpretation and dissemination of study findings and results.

Aim 1: mixed-method context assessment to support UOT adaptation and EFFECT for ASD creation

A sequential mixed-method approach, with secondary quantitative methods preceding primary qualitative methods will be used to identify factors key to the adaptation of UOT and creation of a corresponding implementation plan for use in the mental health services context.

Participants

Participants (N = 100) will initially be recruited for the quantitative survey, with a subset of these participants (N = 40) also recruited to participate in subsequent qualitative focus groups. CAP members and leaders from partnered community-based mental health agencies will be asked to facilitate identification and recruitment of participants. Recruitment will occur through several methods, including in-person appearances at staff meetings, flyers posted in patient areas, and targeted email contact. Survey participants who agree to be contacted regarding future study activities will be recruited to participate in qualitative focus groups. Participants will include mental health leaders (e.g., agency leaders, executive directors), mental health providers (i.e., clinicians), and caregivers of youth with ASD either currently or previously receiving mental health services, namely psychotherapy services.

Procedure

Web-based surveys will be distributed using web-based software. Focus groups will occur at a centralized location identified as most convenient for participants. Separate focus groups will be conducted based on participant type in order to minimize demand characteristics and promote candid response: (1) agency leader (approximately one group), (2) mental health provider (approximately two groups), and (3) caregivers of youth (approximately two groups). Estimated time commitment and compensation for participation include 90 and 20 min and $40 and $20 for focus groups and surveys, respectively.

Measures

Quantitative and qualitative measures will be designed and administered to elicit information regarding inner context (i.e., within organization) targets emphasized in EPIS framework and key to the adaptation of UOT and development of corresponding implementation protocol. Provider and caregiver specific versions will be utilized for quantitative and qualitative measures. Table 1 lists specific survey measures. All established quantitative measures have evidence of strong reliability and validity. Measures will be adapted to assess the nature and impact of difficulties as well as knowledge and confidence with specific executive functioning constructs targeted in UOT (e.g., flexibility, emotional control, planning, and goal setting). Items assessing implementation outcomes (fit, feasibility, utility) will be developed or adapted to assess specific UOT intervention content and format. Semi-structured focus group guides with pre-selected interview questions that correspond to the quantitative surveys will be used to facilitate participant discussion of perspectives regarding: (1) the role and impact of executive functioning in mental health services, (2) executive functioning experience and training needs, and (3) key areas for adaptation of UOT for mental health services, including adaptations to the content, context, and training of UOT and associated provider training.

Table 1 Quantitative measures
Full size table

Data analysis

Informed by the systematic mixed-methods needs assessment of mental health services for youth with ASD in AIM HI [13], a sequential quan → QUAL mixed-methods design will be employed whereby quantitative (hence lower case “quan”) survey data will inform the development and analysis of primary qualitative focus group (hence upper case “QUAL”) [94, 95]. Consistent with current recommendations [94, 96, 97], data will be integrated and triangulated to examine convergence (e.g., do data provide similar answers to the same questions), complementarity (e.g., focus group data providing additional context to aid interpretation of survey data), and expansion (e.g., qualitative data providing explanations or new interpretations for findings produced by survey data). While the design is sequential in that quan data informs the QUAL measures, analyses will involve a bi-directional analysis and interpretation of the two data types.

Aim 2: UOT adaptation and development of corresponding implementation plan to develop EFFECT for ASD

Informed by aim 1 mixed-method findings, an iterative, collaborative approach will be used to adapt and enhance the existing UOT intervention and create a corresponding implementation plan to maximize feasibility and fit in mental health settings.

Participants

Participants include CAP members, the principal investigator, and key mentors and consultants, including original UOT developers.

Procedures

Meetings will focus on providing feedback regarding aim 1 findings as well as discussion and incorporation of findings to guide adaptation of UOT and corresponding implementation plan development and providing feedback on drafted materials. UOT adaptation will include a particular focus on implementation feasibility and external validity of EFFECT for ASD models while also retaining core UOT principles and core components, or those key to UOT effectiveness. Using an iterative approach, meetings will focus on reviewing and refining drafted materials, with particular emphasis on modifications that support implementation feasibility and utility in mental health settings. The principles behind UOT and corresponding intervention components will be outlined, and CAP members will discuss how these might best fit into the mental health service context. Aim 1 results will be presented for validation and used for reference on key modifications. Incorporating this feedback, the principal investigator will develop the initial version of EFFECT for ASD. In subsequent meetings, CAP members will provide feedback for further revisions. This process will be used for the development and revision of all materials and UOT developers and mentors (Drs. Kenworthy and Anthony) will review the protocol throughout this process to assess suggested adaptations and ensure fidelity with the principles and UOT core elements.

This process will likely preserve the UOT components, including modules with multiple EF-focused lessons (e.g., flexibility, planning, self-monitoring, goal setting) and structured lesson plans and materials (e.g., didactic content, application activities). Possible adaptations include (1) modification of language and activities appropriately for the mental health setting (e.g., change language from “classroom” to “session,” adapt to small group or individual activities); (2) adaptation of materials and components key to mental health service context, including those targeting managing challenging behavior [83], parent engagement and skills [98], home generalization and promoting involvement of other key influencers such as other caregivers and teachers; and (3) additional content and format adaptations to better fit providers, caregiver and client needs (e.g., feedback on pattern of executive functioning deficits and appropriate supports, in-person and online tools). See Fig. 2 for a description of the UOT components and proposed adaptation areas informed by the Chambers and Norton’s adaptome platform [99], which systematically captures common areas of EBI adaptations to aid intervention development and implementation efforts. A corresponding implementation plan will be developed and yoked with the adapted intervention and likely include multi-method, evidence-based training [100, 101], biweekly ongoing supervision (practice with feedback), consultation, and fidelity monitoring to aid implementation and fidelity (see aim 3 the “Procedures” section). CAP meetings will occur monthly during this phase lasting from 1-2 h.

Fig. 2
figure 2

Current structure and proposed adaptations of UOT for EFFECT for ASD

Full size image

Measures

Quantitative measures and qualitative approaches will be utilized to characterize and document the adaptations made to UOT and corresponding implementation plan. Measures will be informed by the existing adaptation framework developed by Stirman and colleagues [82] and existing work applying this framework to measure and track adaptions in community-based implementation efforts [102, 103]. Specifically, the Stirman et al., framework will be used to document key aspects of the adaptation process, including how modifications were made, what was modified, at what level of delivery modifications were made, and the type or nature of modifications. See Table 1 for list of aim 2 measures.

Aim 3: feasibility pilot test of EFFECT for ASD in mental health settings

The primary goal of this aim is to conduct a pilot test examining the feasibility of EFFECT for ASD in mental health settings. Pilot study will evaluate (1) feasibility, utility, and perceived fit of EFFECT for ASD in publicly funded mental health service settings; (2) EFFECT use and adherence; and (3) preliminary youth clinical outcomes, including changes in executive functioning and challenging behaviors (see Fig. 3).

Fig. 3
figure 3

Aim 3 EFFECT for ASD consort diagram

Full size image

Participants

Provider-youth dyads (N = 26) will be recruited from partnered community-based mental health clinics. Sample size was informed by prior feasibility pilot studies focused on community-based service interventions for youth and recommendations regarding appropriate statistical analyses [34, 104, 105]. Eligible providers include those employed at one of the three participating clinics for at least 5 months, and have a child with ASD newly assigned to their caseload. Children and their families will be eligible if the child is between the ages of 8-13, has an existing ASD diagnosis according to case records, and are entering a new episode of outpatient mental health care. Facilitated by CAP members, partnered organizational leaders, and eligible providers will be recruited via similar methods as aim 1. Recruitment will occur through several methods, including in-person appearances at staff meetings, flyers posted in patient areas, and targeted email contact. Aim 1 provider participants who consented to be contacted regarding future studies will also be recruited. Youth/family recruitment will be linked to provider recruitment, with interested or enrolled therapists supporting identification and recruitment of eligible youth/family participants through obtaining permission to be contacted by the research team. The research team will subsequently consent and enroll families, resulting in 26 provider-child dyads.

Procedures

All participants will be provided information about the study from research staff and provide consent obtained prior to data collection. Enrolled provider-youth dyads will be randomly assigned to one of two conditions: EFFECT for ASD training and waitlist control. Providers will be randomized by an independent researcher via computer-generated randomization to one of the two study conditions per study (with group assignment based on even versus odd digits). Providers in the treatment condition will receive training according to the protocol developed in aim 2 whereas providers in the waitlist condition will receive training approximately 6 months after those in the treatment condition. Participants will be asked to complete measures pre (baseline) and six-months post implementation of EFFECT for ASD or at the beginning and end of 6 months for the control condition. Across conditions, provider participants will submit video recordings of intervention sessions with enrolled families and a corresponding provider-report fidelity measure to assess intervention fidelity. As with aim 1 findings, CAP will aid with interpretation of findings.

Measures

Quantitative measures and qualitative approaches will be administered to provide a comprehensive understanding of the primary implementation and secondary clinical outcomes of EFFECT for ASD, including feasibility, acceptability, uptake, fidelity, and effectiveness of EFFECT for ASD. Quantitative youth outcome measures will be administered pre and post, whereas implementation outcomes will be collected at post (see Table 1). Fidelity measures which will be assessed throughout EFFECT for ASD implementation. Additional objective metrics of implementation outcomes related to the research process (e.g., enrollment/withdrawal rate, data collection completion rate) will also be examined given prior data highlighting the influence of perceptions regarding research and data collection activities on implementation [67]. For the qualitative focus interviews, semi-structured interview guides with pre-selected questions will be used to gather participant perspectives pertaining to EFFECT for ASD feasibility, acceptability, appropriateness, and effectiveness. As with aim 1, respondent-specific (e.g., provider, caregiver, youth) versions will be administered. See Table 1 for list of all aim 3 measures.

Analyses

Descriptive analyses and examination of qualitative themes related to perceptions of the feasibility, acceptability, and appropriateness of EFFECT for ASD will be used to evaluate key implementation outcomes. For quantitative measures, criteria for determining sufficient feasibility, acceptability, and appropriateness will be applied, including examination of mean-level ratings on relevant measures (see Table 1) and whether mean scores are greater than or equal to ratings indicating perceived feasibility, acceptability, and/or appropriateness (e.g., ≥ 4 on feasibility and acceptability of interventions measures, ≥ 4 on subscales on the perceived characteristics of intervention scale). Scores will also be compared to prior relevant literature. For example, fidelity scores will also be analyzed and compared to prior trials of UOT to determine if providers achieve similar fidelity levels, indicating appropriate uptake and fidelity. Finally and informed by current recommendations and prior work [106], objective measures of outcomes (e.g., recruitment and enrollment, proportion of sessions completed, attrition) will also be examined. Intervention effects analyses will also be conducted, including group (intervention vs. control) × time (pre to post), will be conducted to examine changes in outcome measures over time; analyses will be conducted using recommended procedures (e.g., maximum likelihood) to adjust for missing data and non-normality of outcome variables. Informed by similar pilot feasibility trials, the targeted sample size (N = 26) allows for dropout rates similar to that of prior pilot studies in the same setting while also meeting the threshold for sufficiently precise parameter estimates [34, 104, 105, 107]. Per current recommendations [108], direction of effects and effect size estimation will be examined given the small sample size. Consistent with a type III scale-out [109], this pilot study adapts an evidence-based intervention for a new population and delivery systems and will borrow statistical strength by comparing findings to prior studies regarding evidence of UOT. As such, we anticipate an effect size that approximates the medium effect observed in the UOT randomized-clinical trial [30] (average Cohen’s d = .55). Qualitative and mixed-method analyses of structured interviews will utilize the same procedures as in aim 1.

Discussion

Youth with ASD are a high priority population with significant service needs and challenges, and ASD EBI implementation is limited in community settings. Improving receipt and efficacy of community mental health services will have a positive public health impact through reduction in health care needs and costs. This study aims to improve the effectiveness of community-based services through the development and implementation of a feasible, transdiagnostic intervention targeting potent mechanisms of action underlying ASD and an array of co-occurring mental health problems.

The current project builds directly on prior implementation research within the mental health service context, including those specifically targeting youth with ASD and encompasses several important innovations that will expand the field of implementation science. Specifically, this work expands upon and complements the transdiagnostic approach of AIM HI through the selection of an EBI targeting key transdiagnostic clinical mechanisms that can be used across profiles of mental health disorders within the context of ASD. This serves to increase EBI applicability and coverage for the multiple co-occurring psychiatric conditions in youth with ASD receiving mental health care and address provider-reported training needs while minimizing needed time and resources for training. Also complementary to the development and refinement of AIM HI, this project utilizes a community-partnered approach to systematically adapt an existing EBI and develop an implementation plan within the context of the EPIS framework and a CAP. This approach facilitates the incorporation of inner context factors and intimate, local knowledge critical to maximizing its “fit” with end user and within natural context constraints. This approach of designing and adapting interventions for end-users results in improved clinical and practice outcomes through capitalizing on an existing effectiveness of the intervention while optimizing fit and feasibility and thereby accelerating the implementation and receipt of effective care for this high priority and traditionally challenging population.

Finally, the current study aims to simultaneously test the implementation of EFFECT for ASD in mental health settings and gather pilot data on clinical effectiveness (youth outcomes). This hybrid effectiveness-implementation design [110] builds and expands upon previous UOT effectiveness data and collects data on (1) implementation mechanisms (provider attitudes and intent to use) related to implementation outcomes and (2) clinical mechanisms (changes in executive functioning) related clinical outcomes. Through its focus on key mechanisms of change as well as an EBI that specifically targets a key mechanism consistent with research domain criteria [38], it is responsive to urgent calls and prioritization for further articulating and evaluating mechanisms underlying desired clinical and implementation outcomes [38, 111, 112]. Additionally, this project extends proven techniques to address a new target of treatment (executive functioning) that drives major health, mental health, vocational, and even legal outcomes [113]. Together, these innovations have a strong likelihood of impacting multiple implementation and clinical outcomes, including increased provider EBI adoption and use and improved care effectiveness for youth with ASD, a complex, high priority clinical population.

Next steps

Once developed and pilot tested, findings will inform the development of a larger-scale effectiveness-implementation hybrid type 2 trial of EFFECT for ASD with simultaneous examining EFFECT for ASD clinical effectiveness and implementation plan or strategies. Our primary feasibility and acceptability outcomes will support assessment of readiness for a large-scale trial. Additionally, we are leveraging the focus on scaling out an EBI with good effectiveness and implementation outcomes as well as the use of established approaches known to facilitate the successful development of an effective and feasible intervention in children’s mental health settings to support these next steps. Further large-scale studies will allow for evaluation within the context of broader, more varied mental health organizations or systems providing mental health care to youth with ASD, including evaluation of further multilevel (e.g., system, leader, provider characteristics) factors on EFFECT for ASD adoption and use. Additionally, future studies will also permit examination of methods to optimize EFFECT for ASD, including identification of the smallest effective dose and/or active ingredients of UOT for youth. Finally, the current project is focused specifically on adapting and testing an executive functioning intervention for youth with ASD and co-occurring mental health conditions. However, it is an entry point for generalizable application of EFFECT for ASD to other clinical populations and service settings where executive functioning significantly impacts community-based care for youth. This will help provide community mental health providers with the training they require to address the growing service demands of youth with complex clinical presentations such as youth with ASD and co-occurring mental health conditions.

Trial status

The current project has received Institutional Review Boards approval from San Diego State University as well as partnered health care organizations and broader County Department of Behavioral Health Services. At time of submission, aim 1 quantitative survey data collection has begun.

Availability of data and materials

The application described in this manuscript is freely available. Please contact the lead author for more information.

Abbreviations

AIM HI:

An Individualized Mental Health Intervention for ASD

ASD:

Autism spectrum disorder

CAP:

Community-academic partnership

EBIs:

Evidence-based interventions

EFFECT for ASD:

Executive functioning for enhancing community-based treatment for ASD

EPIS:

Exploration, preparation, implementation, sustainment framework

UOT :

Unstuck and On Target

References

  1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders-5th edition: DSM-5. Arlington: American Psychiatric Association; 2013.

    Book  Google Scholar 

  2. Baio J, Wiggins L, Christensen DL, et al. Prevalence of autism spectrum disorder among children aged 8 years - autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveill Summ. 2018;67(6):1–23. https://doi.org/10.15585/mmwr.ss6706a1.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bui AL, Dieleman JL, Hamavid H, et al. Spending on children’s personal health care in the United States, 1996-2013. JAMA Pediatr. 2016;34(1):150–60. https://doi.org/10.1001/jamapediatrics.2016.4086.

    Article  Google Scholar 

  4. Leigh JP, Du J. Brief report: forecasting the economic burden of autism in 2015 and 2025 in the United States. J Autism Dev Disord. 2015;45(12):4135–9. https://doi.org/10.1007/s10803-015-2521-7.

    Article  PubMed  Google Scholar 

  5. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry. 2003;60(8):837–44. https://doi.org/10.1001/archpsyc.60.8.837.

    Article  PubMed  Google Scholar 

  6. Rosenberg RE, Kaufmann WE, Law JK, Law PA. Parent report of community psychiatric comorbid diagnoses in autism spectrum disorders. Autism Res Treat. 2011:1–10. https://doi.org/10.1155/2011/405849.

  7. Simonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. J Am Acad Child Adolesc Psychiatry. 2008;47(8):921–9. https://doi.org/10.1097/CHI.0b013e318179964f.

    Article  PubMed  Google Scholar 

  8. Horner RH, Carr EG, Strain PS, Todd AW, Reed HK. Problem behavior interventions for young children with autism: a research synthesis. J Autism Dev Disord. 2002;32(5):423–46. https://doi.org/10.1023/A:1020593922901.

    Article  PubMed  Google Scholar 

  9. Mattila ML, Hurtig T, Haapsamo H, et al. Comorbid psychiatric disorders associated with asperger syndrome/high-functioning autism: a community- and clinic-based study. J Autism Dev Disord. 2010;40(9):1080–93. https://doi.org/10.1007/s10803-010-0958-2.

    Article  PubMed  Google Scholar 

  10. Matson JL, Wilkins J, Macken J. The relationship of challenging behaviors to severity and symptoms of autism spectrum disorders. J Ment Health Res Intellect Disabil. 2009;2(1):29–44. https://doi.org/10.1080/19315860802611415.

    Article  Google Scholar 

  11. Brookman-Frazee L, Baker-Ericzén MJ, Stahmer A, et al. Involvement of youths with autism spectrum disorders or intellectual disabilities in multiple public service systems. J Ment Heal Res Intellect Disabil. 2009;2(3):201–19. https://doi.org/10.1080/19315860902741542.Involvement.

    Article  Google Scholar 

  12. Mandell DS, Walrath CM, Manteuffel B, Sgro G, Pinto-Martin J. Characteristics of children with autistic spectrum disorders served in comprehensive community-based mental health settings. J Autism Dev Disord. 2005;35(3):313–21. https://doi.org/10.1007/s10803-005-3296-z.

    Article  PubMed  Google Scholar 

  13. Brookman-Frazee L, Drahota A, Stadnick N, Palinkas LA. Therapist perspectives on community mental health services for children with autism spectrum disorders. Adm Policy Ment Heal. 2012;39(5):365–73. https://doi.org/10.1037/t38826-000.

    Article  Google Scholar 

  14. Montes G, Halterman JS, Magyar CI. Access to and satisfaction with school and community health services for US children with ASD. Pediatrics. 2009;124(Suppl (December)):S407–13. https://doi.org/10.1542/peds.2009-1255L.

    Article  PubMed  Google Scholar 

  15. Brookman-Frazee L, Baker-Ericzén MJ, Stadnick N, Taylor R. Parent perspectives on community mental health services for children with autism spectrum disorder. J Child Fam Stud. 2012;21(4):1–16. https://doi.org/10.1038/nmeth.2250.Digestion.

    Article  Google Scholar 

  16. Stahmer AC, Collings NM, Palinkas LA. Early intervention practices for children with autism: descriptions from community providers. Focus Autism Other Dev Disabl. 2005;20(2):66–79. https://doi.org/10.3816/CLM.2009.n.003.Novel.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wong C, Odom SL, Hume KA, et al. Evidence-based practices for children, youth, and young adults with autism spectrum disorder: a comprehensive review. J Autism Dev Disord. 2015;45(7):1951–66. https://doi.org/10.1007/s10803-014-2351-z.

    Article  PubMed  Google Scholar 

  18. Wood JJ, McLeod BD, Klebanoff S, Brookman-Frazee L. Toward the implementation of evidence-based interventions for youth with autism spectrum disorders in schools and community agencies. Behav Ther. 2015;46(1):83–95. https://doi.org/10.1016/j.beth.2014.07.003.

    Article  PubMed  Google Scholar 

  19. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. https://doi.org/10.1186/1748-5908-8-117.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Chorpita BF, Daleiden EL. Structuring the collaboration of science and service in pursuit of a shared vision. J Clin Child Adolesc Psychol. 2014;43(2):323–38. https://doi.org/10.1080/15374416.2013.828297.

    Article  PubMed  Google Scholar 

  21. Wood JJ, Drahota A, Sze K, Har K, Chiu A, Langer DA. Cognitive behavioral therapy for anxiety in children with autism spectrum disorders: a randomized, controlled trial. J Child Psychol Psychiatry. 2014;50(3):224–34. https://doi.org/10.1111/j.1469-7610.2008.01948.x.Cognitive.

    Article  Google Scholar 

  22. Reaven J, Blakeley-Smith A, Leuthe E, Moody E, Hepburn S. Facing your fears in adolescence: cognitive-behavioral therapy for high-functioning autism spectrum disorders and anxiety. Autism Res Treat. 2012;2012:1–13. https://doi.org/10.1155/2012/423905.

    Article  Google Scholar 

  23. Joshi G, Petty C, Wozniak J, et al. The heavy burden of psychiatric comorbidity in youth with autism spectrum disorders: a large comparative study of a psychiatrically referred population. J Autism Dev Disord. 2010;40(11):1361–70. https://doi.org/10.1007/s10803-010-0996-9.

    Article  PubMed  Google Scholar 

  24. Brookman-Frazee L, Stadnick N, Chlebowski C, Baker-Ericzén MJ, Ganger W. Characterizing psychiatric comorbidity in children with autism spectrum disorder receiving publicly funded mental health services. Autism. 2017;22(8):938–52. https://doi.org/10.1177/1362361317712650.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Chandler RK, Peters RH, Field G, Juliano-Bult D. Challenges in implementing evidence-based treatment practices for co-occurring disorders in the criminal justice system. Behav Sci Law. 2004;22(4):431–48. https://doi.org/10.1002/bsl.598.

    Article  PubMed  Google Scholar 

  26. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004;6(2):61–74. https://doi.org/10.1023/B:MHSR.0000024351.12294.65.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Beidas RS, Edmunds J, Ditty M, et al. Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Adm Policy Ment Heal Ment Heal Serv Res. 2014;41(6):788–99. https://doi.org/10.1038/jid.2014.371.

    Article  CAS  Google Scholar 

  28. Reding MEJ, Chorpita BF, Lau AS, Innes-Gomberg D. Providers’ attitudes toward evidence-based practices: is it just about providers, or do practices matter, too? Adm Policy Ment Heal Ment Heal Serv Res. 2014;41(6):767–76. https://doi.org/10.1007/s10488-013-0525-1.

    Article  Google Scholar 

  29. Iadarola S, Shih W, Dean M, et al. Implementing a manualized, classroom transition intervention for students with ASD in underresourced schools. Behav Modif. 2018;42(1):126–47. https://doi.org/10.1177/0145445517711437.

    Article  PubMed  Google Scholar 

  30. Kenworthy L, Anthony LG, Naiman DQ, et al. Randomized controlled effectiveness trial of executive function intervention for children on the autism spectrum. J Child Psychol Psychiatry Allied Discip. 2014;55(4):374–83. https://doi.org/10.1111/jcpp.12161.

    Article  Google Scholar 

  31. Brookman-Frazee L, Drahota A. An individualized mental health intervention for children with autism spectrum disorders (AIM HI): a model to address challenging behaviors in children with ASD—a therapist manual: The Regents of University of California-San Diego; 2010.

  32. Dyson MW, Chlebowski C, Brookman-Frazee L. Therapists’ adaptations to an intervention to reduce challenging behaviors in children with autism spectrum disorder in publicly funded mental health services. J Autism Dev Disord. 2018;49(3):924–34. https://doi.org/10.1007/s10803-018-3795-3.

    Article  Google Scholar 

  33. Drahota A, Stadnick N, Brookman-Frazee L. Therapist perspectives on training in a package of evidence-based practice strategies for children with autism spectrum disorders served in community mental health clinics. Adm Policy Ment Heal Ment Heal Serv Res. 2014;41(1):114–25. https://doi.org/10.1016/j.immuni.2010.12.017.Two-stage.

    Article  Google Scholar 

  34. Brookman-Frazee L, Drahota A, Stadnick N. Training community mental health therapists to deliver a package of evidence based practice strategies for school-age children with autism spectrum disorders: a pilot study. J Autism Dev Disord. 2012;42(8):1651–61. https://doi.org/10.1007/s10803-011-1406-7.Training.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Brookman-Frazee L, Chlebowski C, Villodas M, Villodas S, Martinez K. Training community therapists to deliver a mental health intervention for ASD: changes in caregiver outcomes and mediating role on child outcomes. J Am Acad Child Adolesc Psychiatry. Under Review.

  36. Brookman-Frazee L, Roesch S, Chlebowski C, Baker-Ericzén MJ, Ganger W. Effectiveness of training therapists to deliver an individualized mental health intervention for children with ASD in publicly funded mental health services. JAMA Psychiatry. 2019;0812:1–10. https://doi.org/10.1001/jamapsychiatry.2019.0011.

    Article  Google Scholar 

  37. Stadnick N, Drahota A, Brookman-Frazee L. Parent perspectives of an evidence-based intervention for children with autism served in community mental health clinics. J Child Fam Stud. 2013;22(3):414–22. https://doi.org/10.1007/s10826-012-9594-0.

    Article  PubMed  Google Scholar 

  38. Insel T, Cuthbert B, Garvey M. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167(7):748–51.

    Article  PubMed  Google Scholar 

  39. Welsh MC, Pennington BF. Assessing frontal lobe functioning in children: views from developmental psychology. 1988;4. https://doi.org/10.1080/87565648809540405.

  40. Snyder HR, Miyake A, Hankin BL. Advancing understanding of executive function impairments and psychopathology: bridging the gap between clinical and cognitive approaches. Front Psychol. 2015;6(MAR). https://doi.org/10.3389/fpsyg.2015.00328.

  41. Barkley RA. Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD. Behav Inhibit Sust Atten Exec Func. 2015;121:65–94. https://doi.org/10.1037/0033-2909.121.1.65.

    Article  Google Scholar 

  42. Dickson KS, Ciesla JAJA, Zelic K. The role of executive functioning in adolescent rumination and depression. Cognit Ther Res. 2016;41(1). https://doi.org/10.1007/s10608-016-9802-0.

  43. Letkiewicz AM, Miller GA, Crocker LD, et al. Executive function deficits in daily life prospectively predict increases in depressive symptoms. Cognit Ther Res. 2014;38(6):612–20. https://doi.org/10.1007/s10608-014-9629-5.

    Article  Google Scholar 

  44. Dickson KS, Ciesla JA. Executive functioning and negative affect: an examination of the mediational effects of emotion regulation. Int J Cogn Ther. 2018;11(3):272–86. https://doi.org/10.1007/s41811-018-0029-6.

    Article  Google Scholar 

  45. Lawson RA, Papadakis AA, Higginson CI, et al. Everyday executive function impairments predict comorbid psychopathology in autism spectrum and attention deficit hyperactivity disorders. Neuropsychology. 2015;29(3):445–53. https://doi.org/10.1037/neu0000145.

    Article  PubMed  Google Scholar 

  46. Hill E. Executive dysfunction in autism. Trends Cogn Sci. 2004;8(1):26–32. https://doi.org/10.1016/j.tics.2003.11.003.

    Article  PubMed  Google Scholar 

  47. Snyder HR, Friedman NP, Hankin BL. Transdiagnostic mechanisms of psychopathology in youth: executive functions, dependent stress, and rumination. Cognit Ther Res. 2019;43(5):834–51. https://doi.org/10.1007/s10608-019-10016-z.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Rosenthal M, Wallace GL, Lawson R, Wills MC, Yerys BE, Kenworthy L. Impairments in real world executive function increase from childhood to adolescence in autism spectrum disorders. Neuropsychology. 2013;27(1):13–8. https://doi.org/10.1037/a0031299.Impairments.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Gilotty L, Kenworthy L, Sirian L, Black DO, Wagner AE. Adaptive skills and executive function in autism spectrum disorders. Child Neuropsychol. 2002;8(4):241–8. https://doi.org/10.1076/chin.8.4.241.13504.

    Article  PubMed  Google Scholar 

  50. Hollocks MJ, Jones CRG, Pickles A, et al. The association between social cognition and executive functioning and symptoms of anxiety and depression in adolescents with autism spectrum disorders. Autism Res. 2014;7(2):216–28. https://doi.org/10.1002/aur.1361.

    Article  PubMed  Google Scholar 

  51. Lieb RW, Bohnert AM. Relations between executive functions, social impairment, and friendship quality on adjustment among high functioning youth with autism spectrum disorder. J Autism Dev Disord. 2017;47(9):1–12. https://doi.org/10.1007/s10803-017-3205-2.

    Article  Google Scholar 

  52. Raver CC, Blair C, Willoughby M. Poverty as a predictor of 4-year-olds’ executive function: new perspectives on models of differential susceptibility. Dev Psychol. 2013;49(2):292–304. https://doi.org/10.1037/a0028343.

    Article  PubMed  Google Scholar 

  53. Hackman DA, Farah MJ. Socioeconomic status and the developing brain Daniel. Trends Cogn Sci. 2009;13(2):65–73. https://doi.org/10.1016/j.tics.2008.11.003.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Burns BJ, Erkanli A, Tweed D, et al. Children’s mental health service use across service sectors. Health Aff. 2003;14(3):147–59. https://doi.org/10.1377/hlthaff.14.3.147.

    Article  Google Scholar 

  55. Eldar S, Ph D, Apter A, et al. Attention bias modification treatment for pediatric anxiety disorders: a randomized controlled trial. Am J Psychiatry. 2012;15:213–20. https://doi.org/10.1176/appi.ajp.2011.11060886.

    Article  Google Scholar 

  56. Klingberg T, Fernell E, Olesen PJ, et al. Computerized training of working memory in children with ADHD — a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry. 2005;44(2):177–86. https://doi.org/10.1097/00004583-200502000-00010.

    Article  PubMed  Google Scholar 

  57. Bettis AH, Coiro MJ, England J, et al. Comparison of two approaches to prevention of mental health problems in college students: enhancing coping and executive function skills. J Am Coll Heal. 2017;65(5):313–22. https://doi.org/10.1080/07448481.2017.1312411.

    Article  Google Scholar 

  58. Cannon L, Kenworthy L, Alexander KC, Werner MA, Anthony LG. Unstuck and on target!: an executive function curriculum to improve flexibility, planning, and organization: Brookes Publishing; 2018.

  59. Wallace GL, Kenworthy L, Pugliese C, et al. Real-world executive functions in adults with autism spectrum disorder: profiles of impairment and associations with adaptive functioning and co-morbid anxiety and depression. J Autism Dev Disord. 2016;46(3):1071–83. https://doi.org/10.1007/s10803-015-2655-7.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Anthony LG, Anthony BJ, Kenworthy L. A community-based executive function intervention for children from low income schools with ADHD and ASD. Final Reserach Report, Patient-Centered Outcomes Research Institute (Peer and Institute Reviewed).; 2019.

  61. Weitlauf AS, McPheeters ML, Peters B, et al. Therapies for children with autism spectrum disorder. Agency for Healthcare Research and Quality (US); 2014. http://www.ncbi.nlm.nih.gov/pubmed/25210724. Accessed 19 Sept 2017.

  62. Kenworthy L, Anthony LG, Hardy KK, et al. Addressing disparities comparative effectiveness trial of community-based executive function treatments in ASD & ADHD. In: Presentation at the annual meeting of the international society for autism, ed. San Francisco; 2017.

  63. Anthony LG, Verbalis A, Pugliese C, et al. What matters?: an examination of implementation factors in the effectiveness of two tier 2 school-based interventions. In: Kenworthy L, editor. Reaching children where they are: the promise and challenge of school-based intervention for autism. Rotterdam, Netherlands: Panel Presentation at the Annual Meeting of the International Society for Autism Research; 2018.

    Google Scholar 

  64. Anthony LG, Anthony BJ, Verbalis A, et al. An exploration of possible moderators in an addressing disparities comparative effectiveness trial for elementary students with ASD or ADHD. In: Gruber A, editor. Psychosocial and demographic moderators of intervention outcomes for youth and young adults with ASD. Rotterdam: Panel Presentaiton at the Annual Meeting of the International Society for Autism Research; 2018.

    Google Scholar 

  65. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Heal Ment Heal Serv Res. 2011;38(1):4–23. https://doi.org/10.1007/s10488-010-0327-7.

    Article  Google Scholar 

  66. Moullin JC, Dickson KS, Stadnick N, Rabin BA, Aarons GA. Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implement Sci. 2019;14(1). https://doi.org/10.1186/s13012-018-0842-6.

  67. Brookman-Frazee L, Chlebowski C, Suhrheinrich J, et al. Characterizing shared and unique implementation influences in two community services systems for autism: applying the EPIS framework to two large-scale autism intervention community effectiveness trials. Adm Policy Ment Heal Ment Heal Serv Res. 2019;47:176–87. https://doi.org/10.1007/s10488-019-00931-4.

    Article  Google Scholar 

  68. Lim A, Nakamura BJ, Higa-McMillan CK, Shimabukuro S, Slavin L. Effects of workshop trainings on evidence-based practice knowledge and attitudes among youth community mental health providers. Behav Res Ther. 2012;50(6):397–406. https://doi.org/10.1016/j.brat.2012.03.008.

    Article  PubMed  Google Scholar 

  69. Williams JR, Williams WO, Dusablon T, et al. Evaluation of a randomized intervention to increase adoption of comparative effectiveness research by community health organizations. J Behav Heal Serv Res. 2014;41(3):308–23. https://doi.org/10.1007/s11414-013-9369-4.

    Article  Google Scholar 

  70. Ruzek JI, Eftekhari A, Crowley J, Kuhn E, Karlin BE, Rosen CS. Post-training beliefs, intentions, and use of prolonged exposure therapy by clinicians in the veterans health administration. Adm Policy Ment Heal Ment Heal Serv Res. 2015;44(1):123–32. https://doi.org/10.1007/s10488-015-0689-y.

    Article  Google Scholar 

  71. Kotte A, Hill KA, Mah AC, et al. Facilitators and barriers of implementing a measurement feedback system in public youth mental health implementation facilitators and barriers of assessment strategies. Adm Policy Ment Heal. 2016;43:861–78. https://doi.org/10.1007/s10488-016-0729-2.

    Article  Google Scholar 

  72. Willging CE, Lamphere L, Rylko-Bauer B. The transformation of behavioral healthcare in New Mexico. Adm Policy Ment Heal Ment Heal Serv Res. 2015;42(3):343–55. https://doi.org/10.1007/s10488-014-0574-0.

    Article  Google Scholar 

  73. Palinkas LA, Schoenwald SK, Hoagwood K, Landsverk JA, Chorpita BF, Weisz JR. An ethnographic study of implementation of evidence-based treatments in child mental health: first steps. Psychiatr Serv. 2008;59(7):736–46. https://doi.org/10.1176/appi.ps.59.7.738.112.

    Article  Google Scholar 

  74. Drahota A, Meza RD, Brikho B, et al. Community-academic partnerships: a systematic review of the state of the literature and recommendations for future research. Milbank Q. 2016;94(1):163–214. https://doi.org/10.1111/1468-0009.12184.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Brookman-Frazee L, Stahmer A, Stadnick N, Chlebowski C, Herschell AD, Garland AF. Characterizing the use of research-community partnerships in studies of evidence-based interventions in children’s community services. Adm Policy Ment Heal Ment Heal Serv Res. 2016;43(1):93–104. https://doi.org/10.1007/s10488-014-0622-9.

    Article  Google Scholar 

  76. Garland AF, Brookman-Frazee L. Therapists and researchers: advancing collaboration. Psychother Res. 2015;25(1):95–107. https://doi.org/10.1080/10503307.2013.838655.

    Article  PubMed  Google Scholar 

  77. Brookman-Frazee L, Stahmer AC, Lewis K, Fedor JD, Reed S. Building a research-community collaborative to improve community care for infants and toddlers at-risk for autism spectrum disorders. J Community Psychol. 2012;40(6):715–34. https://doi.org/10.1002/jcop.21501.BUILDING.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Lau AS, Brookman-Frazee L. The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services. Implement Sci. 2016;11(1):31. https://doi.org/10.1186/s13012-016-0388-4.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Lyon AR, Charlesworth-Attie S, Vander SA. Modular psychotherapy for youth with internalizing based health centers. 2011;40(4):569–81.

  80. Beidas RS, Stewart RE, Adams DR, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system HHS public access. Adm Policy Ment Heal. 2016;43(6):893–908. https://doi.org/10.1007/s10488-015-0705-2.

    Article  Google Scholar 

  81. Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S. Introduction gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof. 2007;27(2):94–102. https://doi.org/10.1002/chp.106.

    Article  PubMed  Google Scholar 

  82. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1). https://doi.org/10.1186/s13012-019-0898-y.

  83. Brookman-Frazee L, Taylor R, Garland AF. Characterizing community-based mental health services for children with autism spectrum disorders and disruptive behavior problems. J Autism Dev Disord. 2010;40(10):1188–201. https://doi.org/10.1007/s10803-010-0976-0.

    Article  PubMed  PubMed Central  Google Scholar 

  84. Moullin JC, Ehrhart MG, Aarons GA. Development and testing of the measure of innovation-specific implementation intentions (MISII) using Rasch measurement theory. Implement Sci. 2018;13(1). https://doi.org/10.1186/s13012-018-0782-1.

  85. Weintraub S, Dikmen SS, Heaton RK, et al. Cognition assessment using the NIH toolbox. Neurology. 2013;80(11, Supplement 3):S54–64. https://doi.org/10.1212/WNL.0b013e3182872ded.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Gioia GA, Isquith PK, Kenworthy L. Behavior rating inventory of executive function. Lutz: Psychological Assessment Resources; 2000.

    Google Scholar 

  87. Wechsler D. Weschler abbreviated scale of intelligence. San Antonio; 1999.

  88. Kenworthy L, Freeman A, Ratto AB, et al. The executive function challenge task (EFCT): a performance-based observational measure of flexibility and planning in youth. Under Review.

  89. Cook JM, Thompson R, Schnurr PP. Perceived characteristics of intervention scale: development and psychometric properties. Assessment. 2015;22(6):704–14. https://doi.org/10.1177/1073191114561254.

    Article  PubMed  Google Scholar 

  90. Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):1–12. https://doi.org/10.1186/s13012-017-0635-3.

    Article  Google Scholar 

  91. Eyeberg SM, Pincus D. Eyberg child behavior inventory and Sutter-Eyberg student behavior inventory - revised: professional manual. Lutz: Psychological Assessment Resources; 1999.

    Google Scholar 

  92. Jellinek MS, Murphy JM, Robinson J, Feins A, Lamb S, Fenton T. Pediatric symptom checklist: screening school-age children for psychosocial dysfunction. J Pediatr. 1988;112(2):201–9. https://doi.org/10.1016/S0022-3476(88)80056-8.

    Article  CAS  PubMed  Google Scholar 

  93. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles: an integrated system of multi-informant assessment. Burlington; 2001.

  94. Palinkas LA, Horwitz SM, Chamberlain P, Hurlburt MS, Landsverk J. Mixed-methods designs in mental health services research: a review. Psychiatr Serv. 2011;62(3):255–63.

    Article  PubMed  Google Scholar 

  95. Brown CH, Curran GM, Palinkas LA, et al. An overview of research and evaluation designs for dissemination and implementation. Ssrn. 2017;38(1):1–22. https://doi.org/10.1146/annurev-publhealth-031816-044215.

    Article  Google Scholar 

  96. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Heal Ment Heal Serv Res. 2011;38(1):44–53. https://doi.org/10.1007/s10488-010-0314-z.

    Article  Google Scholar 

  97. Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA. Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services. Child Maltreat. 2012;17(1):67–79. https://doi.org/10.1177/1077559511426908.

    Article  PubMed  Google Scholar 

  98. Haine-Schlagel R, Walsh NE. A review of parent participation engagement in child and family mental health treatment. Clin Child Fam Psychol Rev. 2015;18(2):133–50. https://doi.org/10.1007/s10567-015-0182-x.

    Article  PubMed  PubMed Central  Google Scholar 

  99. Chambers DA, Norton WE. The adaptome. Advancing the science of intervention adaptation. Am J Prev Med. 2016;51:124–31. https://doi.org/10.1016/j.amepre.2016.05.011.

    Article  Google Scholar 

  100. Powell BJ, McMillen JC, Proctor EK, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57. https://doi.org/10.1177/1077558711430690.

    Article  PubMed  Google Scholar 

  101. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30(4):448–66. https://doi.org/10.1016/j.cpr.2010.02.005.

    Article  PubMed  PubMed Central  Google Scholar 

  102. Lau AS, Stadnick N, Regan J, et al. Therapist report of adaptations to delivery of evidence-based practices within a system-driven reform of publicly funded children’s mental health services. J Consult Clin Psychol. 2017;85(7):664–75.

    Article  PubMed  PubMed Central  Google Scholar 

  103. Rabin BA, McCreight M, Battaglia C, et al. Systematic, multimethod assessment of adaptations across four diverse health systems interventions. Front Public Heal. 2018;6(April). https://doi.org/10.3389/fpubh.2018.00102.

  104. Haine-Schlagel R, Martinez JI, Roesch SC, Bustos CE, Janicki C. Randomized trial of the parent and caregiver active participation toolkit for child mental health treatment. J Clin Child Adolesc Psychol. 2016;4416(July):1–11. https://doi.org/10.1080/15374416.2016.1183497.

    Article  Google Scholar 

  105. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4(4):287–91. https://doi.org/10.1002/pst.185.

    Article  Google Scholar 

  106. Eldridge SM, Chan CL, Campbell MJ, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355(i5239). https://doi.org/10.1136/bmj.i5239.

  107. Burrell TL, Postorino V, Scahill L, et al. Feasibility of group parent training for children with autism spectrum disorder and disruptive behavior: a demonstration pilot. J Autism Dev Disord. 2020;Advanced Online Publication. https://doi.org/10.1007/s10803-020-04427-1.

  108. Leon AC, Davis LL, Kraemer HC. Role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2012;45(5):626–9. https://doi.org/10.1016/j.jpsychires.2010.10.008.The.

    Article  Google Scholar 

  109. Aarons GA, Sklar M, Mustanski B, Benbow N, Brown CH. “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implement Sci. 2017;12(1):111. https://doi.org/10.1186/s13012-017-0640-6.

    Article  PubMed  PubMed Central  Google Scholar 

  110. Curran GM, Bauer MS, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. https://doi.org/10.1097/MLR.0b013e3182408812.

    Article  PubMed  PubMed Central  Google Scholar 

  111. National Institute of Health. PAR-19-274: dissemination and implementaiton research in health (R01).; 2019. https://grants.nih.gov/grants/guide/pa-files/PAR-18-007.html.

  112. Lewis CC, Klasnja P, Powell BJ, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Heal. 2018;6(May). https://doi.org/10.3389/fpubh.2018.00136.

  113. Moffitt TE, Arseneault L, Belsky D, et al. A gradient of childhood self-control predicts health, wealth, and public safety. Proc Natl Acad Sci U S A. 2011;108(7):2693–8. https://doi.org/10.1073/pnas.1010076108.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We are grateful for the support and collaboration from our health care organizational and community partners, including the providers, youth and families participating in the study. We would like to acknowledge our colleagues, project staff, community partners, families who provided feedback on the study design, and recruitment procedures and research staff time to carry out recruitment activities.

Funding

This work was supported by grants from the National Institute of Mental Health (K23MH115100; PI: Dickson). Additionally, Drs. Dickson and Brookman-Frazee are fellows and Dr. Aarons is core faculty with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis, through an award from the National Institute of Mental Health (R25 MH080916-08).

Author information

Authors and Affiliations

  1. San Diego State University, San Diego, CA, USA

    Kelsey S. Dickson

  2. Child and Adolescent Services Research Center, San Diego, CA, USA

    Kelsey S. Dickson, Gregory A. Aarons & Lauren Brookman-Frazee

  3. Department of Psychiatry, University of California San Diego, La Jolla, CA, USA

    Gregory A. Aarons, Katherine Williams & Lauren Brookman-Frazee

  4. University of Colorado School of Medicine, Aurora, CO, USA

    Laura Gutermuth Anthony

  5. Children’s Hospital of Colorado, Aurora, CO, USA

    Laura Gutermuth Anthony

  6. Center for Autism Spectrum Disorders, Children’s National, Washington, DC, USA

    Lauren Kenworthy

  7. Rady Children’s Hospital-San Diego, San Diego, CA, USA

    Brent R. Crandal & Katherine Williams

  8. Autism Disovery Institute at Rady Children’s Hospital, San Diego, CA, USA

    Lauren Brookman-Frazee

Authors
  1. Kelsey S. Dickson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gregory A. Aarons
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laura Gutermuth Anthony
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lauren Kenworthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brent R. Crandal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Katherine Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lauren Brookman-Frazee
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KD is the principal investigator for the study protocol, developed the study concept and design, served as the primary writer of the manuscript, and approved all changes. GA and LBF are the co-primary mentors and LGA and LK are mentors for the K23 funding (PI: Dickson), who provide research and training support for all study activities. All authors were involved in developing, editing, reviewing, and providing feedback for this manuscript and have given approval of the final version to be published.

Corresponding author

Correspondence to Kelsey S. Dickson.

Ethics declarations

Ethics approval and consent to participate

San Diego State University (protocol # HS-2019-0147), Rady Children’s Hospital (project # 4245) and County of San Diego Department of Behavioral Health Services Institutional Review Boards approved study activities related to aim 1.

Consent for publication

Not applicable.

Competing interests

All authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dickson, K.S., Aarons, G.A., Anthony, L.G. et al. Adaption and pilot implementation of an autism executive functioning intervention in children’s mental health services: a mixed-methods study protocol. Pilot Feasibility Stud 6, 55 (2020). https://doi.org/10.1186/s40814-020-00593-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40814-020-00593-2

Keywords

Pilot and Feasibility Studies

ISSN: 2055-5784