Skip to main content
Download PDF

Leveraging behavioral economics and implementation science to engage patients at risk for suicide in mental health treatment: a pilot study protocol

Pilot and Feasibility Studies volume 8, Article number: 181 (2022) Cite this article

Abstract

Background

Primary care is an ideal setting to connect individuals at risk for suicide to follow-up care; however, only half of the patients referred from the primary care attend an initial mental health visit. We aim to develop acceptable, feasible, low-cost, and effective new strategies to increase treatment initiation among at-risk individuals identified in primary care.

Methods

We will conduct a multi-phase, mixed-methods study. First, we will conduct a chart review study by using administrative data, including medical records, to identify characteristics of primary care patients at risk for suicide who do or do not attend an initial mental health visit following a referral. Second, we will conduct a mixed methods study by using direct observations and qualitative interviews with key stakeholders (N = 65) to understand barriers and facilitators to mental health service initiation among at-risk individuals. Stakeholders will include patients with suicidal ideation referred from primary care who do and do not attend a first mental health visit, primary care and behavioral health providers, and individuals involved in the referral process. We also will collect preliminary self-report and behavioral data regarding potential mechanisms of behavior change (i.e., self-regulation and social support) from patients. Third, we will leverage these findings, relevant frameworks, and the extant literature to conduct a multi-arm, non-randomized feasibility trial. During this trial, we will rapidly prototype and test strategies to support attendance at initial mental health visits. Strategies will be developed with subject matter experts (N = 10) and iteratively pilot tested (~5 patients per strategy) and refined. Research will be completed in the Penn Integrated Care Program (PIC), which includes fourteen primary care clinics in Philadelphia that provide infrastructure for electronic referrals, patient communication, and data access.

Discussion

We will leverage frameworks and methods from behavioral economics and implementation science to develop strategies to increase mental health treatment initiation among individuals at risk for suicide identified in primary care. This project will lead to an evaluation of these strategies in a fully powered randomized trial and contribute to improvements in access to and engagement in mental health services for individuals at risk for suicide.

Trial registration

ClinicalTrials.gov Identifier: NCT05021224

Peer Review reports

Background

Suicide is a major global public health problem. In 2018, there were approximately 800,000 suicide deaths worldwide [1] and over one million people attempt suicide each year [2]. The effects of suicide are far reaching, affecting family members, friends, and communities, and costing the USA more than $90 billion annually [3]. Despite increased attention to suicide as a significant public health concern, current suicide rates are approximately 30% higher than at the turn of the millennium [4]. Although evidence-based practices (EBPs) for suicide prevention exist and individuals are less likely to die by suicide if they engage in services [5,6,7], individuals at risk for suicide often have difficulty engaging in mental health treatment. To date, most research in this area has examined mental health treatment initiation following acute services for patients at risk for suicide [8, 9]. Studies have not yet developed or tested strategies to increase mental health treatment initiation among individuals at risk for suicide identified in primary care, even though a large proportion of these individuals do not go on to initiate mental health treatment following referral [10,11,12].

Primary care is a logical place to identify and connect individuals at risk for suicide to follow-up care because most adults visit primary care annually [13]. Approximately two thirds of people who die by suicide interact with a primary care clinician in the year prior to death [14]. While mental health screening and referral are common in primary care, even when patients are referred to a mental health provider within the same practice only half attend an initial visit [15, 16]. Strategies such as reminder calls and texts, motivational and informational interventions, and case management have only demonstrated small to moderate effects [17]. Even when these interventions are implemented, about 40% of patients do not initiate treatment [17], indicating the need for more effective methods for increasing treatment initiation.

Behavioral science theories and methods hold promise for improving patient engagement [18]. Behavioral economics, which combines classic economic theory with insights from psychology to understand common, predictable factors that influence decision making [19], can be leveraged to improve treatment engagement [20]. Applying behavioral insights effectively requires two steps. First, it is essential to understand the target problem, context, and behavioral and structural barriers and facilitators. Principles from implementation science provide guidance for achieving this understanding and can enhance knowledge of the ways in which organizational factors and stakeholder perspectives influence innovation adoption [21, 22]. Second, it is important to understand key mechanisms that underlie changes in behavior. The Science of Behavior Change (SOBC) provides an experimental therapeutic framework that guides the identification of mechanisms of change to better understand how and why certain strategies work [23]. Factors related to self-regulation (e.g., temporal discounting) and interpersonal and social processes (e.g., emotional social support) may impact the decision to initiate treatment. Thus, it is important to target putative mechanisms, such as temporal discounting and social support, when designing strategies and to attend to contextual factors that impact implementation success (see Fig. 1).

Fig. 1
figure 1

Conceptual model

Full size image

Relevant frameworks

The EAST framework

As noted above, this study draws on principles from behavioral economics, implementation science, and experimental therapeutics. Behavioral economists have identified a wide range of heuristics in human decision making (i.e., cognitive biases), many of which contribute to poor health outcomes. The EAST Framework asserts that, to encourage behavior change and reduce people’s reliance on cognitive biases, the desired behavior should be Easy, Attractive, Social, and Timely [24]. The first of the EAST principles, make it Easy to change behavior, includes strategies such as utilizing default options (e.g., auto enrollment in 401k), reducing the effort required to perform an activity, and simplifying messaging. The second principle is to make the desired behavior Attractive (e.g., use of attention-grabbing messaging and incentives). The third principle, make it Social, includes demonstrating that most people engage in the desired behavior, harnessing the power of networks, and encouraging people to express commitment to others. The final principle, make it Timely, focuses on careful consideration of when to implement strategies, such as providing prompts when people are most likely to be receptive or engaging in collaborative planning for action.

Four steps are recommended to apply these insights [24]: (1) define the outcome (i.e., increase attendance at first mental health visit), (2) understand the context (i.e., what stops people from attending), (3) build the intervention (i.e., develop implementation strategies), and (4) test, learn, and adapt (e.g., evaluate implementation strategies and engage in rapid prototyping). Principles from both experimental therapeutics and implementation science can be leveraged to execute these steps with maximum effectiveness and inform our conceptual model (Fig. 1). For example, in order to understand the context (step 2), it is essential to examine why people have difficulty engaging in a desired behavior (i.e., the context) and why or how an intervention works to change behavior (i.e., mechanisms). Principles and methods from implementation science, such as conducting contextual inquiries to identify barriers and facilitators to evidence-based practice implementation, can be applied to better understand the context and determinants affecting treatment engagement.

The science of behavior change (SOBC) framework

The SOBC Common Fund Program, which applies an experimental therapeutics approach to behavior change, has identified key domains of mechanisms relevant to multiple health behaviors and clinical outcomes that informed our conceptual model and measurement strategy [23]. Mechanisms, critical to advancing the science of implementation, refer to “processes that are responsible for change” or the specific way by which implementation strategies affect implementation success and patient outcomes. Identification of mechanisms can lead to more efficient, effective, and tailored strategies based on the practice of interest and the implementation context [25].

Self-regulation and interpersonal and social processes, two SOBC domains, may mediate the relationship between engagement strategies and treatment initiation. Within the domain of self-regulation, temporal discounting—the cognitive process of discounting rewards that are not immediate—may be a relevant mechanism (e.g., choosing to turn off the alarm clock to gain the immediate reward of sleeping versus attending an initial therapy appointment, which promises the future reward of symptom reduction). A potential strategy to target this mechanism might be a financial incentive to attend treatment sessions [26]. A second putative SOBC mechanism is social-emotional processes (e.g., emotional social support), one type of interpersonal, or social process. Caring Contacts, for example, are brief, non-demanding letters or texts expressing care or concern that have been associated with decreases in suicidal ideation and deaths and may target this mechanism [27, 28].

The promise of rapid prototyping

To address this public health crisis, we must expedite the rate at which innovations to engage individuals at risk for suicide in treatment enter routine practice [29, 30]. Traditionally, intervention effectiveness is tested by expensive, lengthy trials, followed by dissemination and implementation activities. Rapid prototyping involves a series of brief, rigorous tests to optimize operations in early-study stages. Using rapid prototyping increases the likelihood that interventions tested in trials will be acceptable, feasible, and effective, thereby shortening the process of developing and implementing interventions. Industries outside of health care commonly use this approach to learn quickly and “de-risk” decision-making prior to a large rollout [31, 32].

Leveraging the work of the Penn ALACRITY (Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness) Center for transforming mental health delivery through behavioral economics and implementation science [33], we will develop, pilot, and optimize strategies to increase treatment initiation for persons at risk for suicide and accelerate the pace at which promising strategies enter routine care. We will first develop strategies in collaboration with a team of experts in suicide, implementation science, and behavioral economics. We will then iteratively pilot test and refine these preliminary strategies using rapid prototyping. Through this process, we will uncover additional barriers and facilitators to implementing strategies that will allow us to further refine and optimize them prior to testing them in a future randomized trial.

Aims

The main objective of this study is to develop acceptable, feasible, low-cost, and effective strategies that increase patients’ treatment initiation (i.e., attendance at a first mental health visit) following identification of suicide risk in primary care. To complete this objective, we will partner with a large and diverse health system to conduct a multiphase, mixed-methods study (see Fig. 2). We will first conduct a chart review study to identify characteristics of patients at risk for suicide who do or do not attend an initial mental health visit following a referral from primary care and variation in patient attendance among referring providers and practices (Aim 1). We will then conduct a mixed methods study by using qualitative interviews and direct observations with stakeholders to identify barriers and facilitators to mental health treatment attendance for individuals at risk for suicide (Aim 2). In accordance with an experimental therapeutics approach to behavior change, we will also collect preliminary data on mechanisms of action that may impede treatment attendance (i.e., self-regulation and social support) from patients using self-report and behavioral measures [34,35,36,37]. Finally, using procedures from implementation science [38] and behavioral economics [39], we will leverage the information gleaned from Aims 1 and 2, relevant theories and frameworks (e.g., EAST [24];), and the extant literature to develop preliminary strategies to support attendance at first mental health visits. We will then conduct a multi-arm, non-randomized feasibility trial to rapidly prototype and test strategies to optimize engagement by using a series of rigorous, iterative tests (Aim 3). To maximize generalizability, we will develop strategies for increasing attendance in both collaborative care and outpatient specialty mental health.

Fig. 2
figure 2

Study aims and design for multiphase, mixed-methods study

Full size image

Method

Setting

In 2018, a collaborative care program was developed through a partnership between Penn’s Department of Psychiatry and Primary Care Service Line and is referred to as the Penn Integrated Care (PIC) program (a detailed description is provided by Wolk and colleagues [40];). Fourteen primary care clinics in Philadelphia and the surrounding counties that range in size and include family medicine and internal medicine practices have PIC services embedded in their clinics. These practices provide infrastructure for electronic referrals, patient communication, and data access. All practices use a sophisticated electronic health record (EHR) system and patient registries that enables the capture of episodes of care and billing for services using Collaborative Care billing codes [41]. A master’s level mental health clinician is embedded within each practice and delivers traditional Collaborative Care services in collaboration with a consulting psychiatrist [42], as well as brief evidence-based interventions, including behavioral activation, motivational interviewing, and problem-solving therapy, for those with mild to moderate mental illness [40]. The PIC model also includes a centralized “Resource Center” staffed by bachelors-level mental health intake coordinators who are supervised by licensed clinicians. This Resource Center assesses, triages, and refers primary care patients in need of mental health services to the health system and community providers.

Aim 1 (chart review study)

Aim 2 is to identify the characteristics of patients at risk for suicide who do or do not attend an initial mental health visit following referral from primary care and variation in patient attendance among referring providers and practices.

Sample

The sample will consist of adults referred for mental health services from primary care in PIC practices, including all Medicare, Medicaid, and commercial insurance enrollees. Based on preliminary data, we anticipate that our study sample will include 916 adults with a score of ≥1 on the PHQ-9 suicide item who were referred to PIC between 2018 and mid-2021. By comparison, there were 2470 adults with a score of ≥1 on the PHQ-9 suicide item who were referred to specialty care and/or did not have insurance coverage for PIC.

Procedure

We will use administrative data, including medical records and insurance claims data, to identify which patient characteristics (e.g., diagnoses, demographics) are associated with attendance at mental health visits and examine variations in patient attendance among referring providers and practices. Our primary data source will be medical records; when available, we will also use insurance claims data. These analyses are exploratory.

Measures

We will rely primarily on medical records to examine treatment engagement among patients with suicidal ideation or behaviors. Treatment engagement will be measured by attendance at a mental health visit following a referral. For a subset of patients, we will review insurance claims data to examine whether they utilized specialty care outside of Penn Medicine. The patients selected will depend on access to insurance claims and approval to analyze these claims.

Medical records

Medical records will be de-identified and sent securely to our study team by program leadership and the Data Analytics Center at Penn Medicine. They will include patient demographic and clinical information, including age, gender, race/ethnicity, insurance type, any utilization of care at Penn Medicine (e.g., specialty care, hospitalizations), prescriptions, patient acuity (on a 1–4 scales), and behavioral health screenings (e.g., PHQ-9 for depression, GAD-7 for anxiety [43, 44];).

Insurance claims

Insurance claims will provide similar demographic and utilization data as medical records but will additionally include any care utilization occurring outside of Penn Medicine. We anticipate that insurance claims will be accessible for Medicaid enrollees from Philadelphia and/or enrollees of one major commercial insurer.

Analyses

We will compare characteristics across (1) patients who screen positive for suicidal ideation (have a score of ≥1 on item 9 of the PHQ-9, which is administered at least annually to all Penn patients), and attend the first mental health visit following referral [43], to (2) patients who screen positive for suicidal ideation and do not attend the first mental health visit following referral. We will detect differences in group means using t tests, then use logistic regressions to identify characteristics that are associated with engagement.

Our equation of interest is Ai = α + Xiβ + εI, where A is a binary variable that indicates whether patient i attended their initial follow-up appointment after the suicide screening. The vector X includes covariates, including age, gender, and race/ethnicity. The unit of analysis is the patient level. We will cluster standard errors at the clinic level and assess model appropriateness and fit using the relevant diagnostics.

We will also compare the two groups’ service use by examining outpatient, inpatient, and pharmacological treatment in the year prior to screening. For patients with accessible insurance claims, we will measure service utilization after the PHQ-9 screen to track mental health treatment engagement outside of Penn Medicine [43]. This will serve as a validity check of medical records to measure engagement because the claims will capture relevant primary or specialty care accessed outside of Penn Medicine.

Aim 2 (mixed methods study)

Aim 2 is to use qualitative interviews and direct observations with stakeholders, as well as self-report and behavioral measures from patients, to identify barriers and facilitators to mental health treatment attendance for individuals at risk of suicide.

Sample

Following the initial inspection of the EHR data in Aim 1, we will identify patients referred from Penn primary care to mental health services following a positive screen for suicidal ideation on the PHQ-9 who did or did not attend their scheduled mental health visit over the past 3 months [43]. All patients must (1) be 18 years and older, able to communicate in English, and able and willing to provide informed consent and (2) present with elevated suicidal ideation per item 9 of the PHQ-9 (item score ≥ 1) completed during a primary care visit. Exclusion criteria include (1) an imminent risk of suicide (i.e., plan with the method and intent to act) and (2) currently experiencing a psychotic episode requiring emergency services and/or precluding abilities to provide informed consent.

Additionally, we will interview primary care and behavioral health providers, as well as individuals involved in the implementation of integrated care at Penn. We will include Penn primary care providers who have referred one or more patients with a positive screen for suicidal ideation for mental health services over the past 3 months. Behavioral health providers will include those practicing at Penn and in community clinics to which Penn patients are routinely referred. Once we identify patients and providers, we will randomly select individuals to invite to participate in qualitative interviews. Individuals involved in the implementation of integrated care at Penn include leaders in Primary Care and Psychiatry, supervisors of behavioral health providers, and mental health intake coordinators. All providers, leaders, and intake coordinators must (1) be 18 years and older, able to communicate in English, and able and willing to provide informed consent and (2) be employed in an appropriate primary care or behavioral health role.

We anticipate conducting a total of 65 interviews including with patients who do (n = 12) and do not (n = 12) attend a first mental health visit, primary care providers (n = 12), behavioral health providers (n = 12), intake coordinators (n = 5), and leaders (n = 12). While we anticipate conducting 65 interviews, we will continue until saturation is achieved.

Procedure

Guided by EAST, we will use observation and semi-structured qualitative interviews with key informants to understand stakeholder experiences and perspectives around mental health treatment engagement. Interviews will include questions about barriers and facilitators of treatment initiation and attendance. For patient stakeholders, we also will assess self-regulation and social support with self-report and behavioral measures. Participants will be compensated $30 an hour for participating. Finally, we will observe intake coordinators in the Resource Center as they assess and schedule patients with mental health providers. Although we are underpowered to detect significant effects, we hypothesize that lower self-regulation and lower social support will predict patient non-attendance at initial mental health visits.

Measures (Table 1)

Self-regulation

We will measure self-regulation, specifically temporal discounting, using two SOBC-recognized measures: (1) The Kirby Delay-Discounting Task (KDDT) [34] and (2) The Consideration of Future Consequences (CFC) Scale [36].

Table 1 Alignment of measures with conceptual model
Full size table

The KDDT is a behavioral task that assesses the tendency to devalue rewards as they become more temporally remote (known as delay-discounting) that has demonstrated reliability and validity [34, 35, 45]. The KDDT has 27 items that provide two choices for each question: an immediate reward or a larger but delayed reward (e.g., $19 today or $25 in 53 days). Acceptable test-retest reliability has been established at 5 weeks (r = .77) and 1 year (r = .71; 37).

The CFC scale is a 12-item self-report measure of consideration of future consequences or forward-thinking behavior [36]. Each CFC item (e.g., “Often I engage in a particular behavior in order to achieve outcomes that may not result for many years”) is rated on a 1-5 Likert-type scale ranging from “extremely uncharacteristic” to “extremely characteristic.” The total score ranges from 12 to 60, with higher scores indicating greater consideration of future consequences. The CFC has demonstrated strong internal consistency (Cronbach’s alpha .80–.86) and acceptable test-retest reliability (r = .72) [36].

Interpersonal processes (emotional social support)

We will measure perceived emotional social support using the 12-item Multidimensional Scale of Perceived Social Support (MSPSS) [37]. The MSPSS, a self-report measure recognized by the Research Domain Criteria Project (RDoC), is rated on a 1–7 Likert-type scale ranging from “Very Strongly Disagree” to “Very Strongly Agree” in response to each statement (e.g., “I get the emotional help and support I need from my family”). The total score ranges from 12 to 84, with higher scores indicating greater perceived social support. Internal consistency (Cronbach’s alpha = .88) and test-retest reliability (r = .85) are well established [37]. Divergent validity of the MSPSS has also been established; the measure has been found to be inversely correlated with measures of depression and anxiety [46].

Structural barriers and provider attitudes (qualitative interviews)

Individual, semi-structured interviews will be conducted to understand cognitive biases and contextual factors that may impact treatment engagement broadly and attendance at initial mental health visits in particular. We developed interview guides for each stakeholder group to ensure uniform inclusion and sequencing of topics and to allow for valid comparison across interviews. The first section asks about general views related to mental health services, including perceptions of (1) what constitutes mental health care, (2) why mental health care is or is not important, (3) barriers and facilitators to attending mental health visits, (4) use of and experience with mental health services, and (5) engagement strategies used or recommended (for providers, intake coordinators, leaders).

The second section asks about relevant cognitive biases, including (1) what is appealing and satisfactory about mental health care, (2) what is unappealing and unsatisfactory about mental health care, and (3) perceived expectations of others related to mental health care. Finally, patients will be asked to recount the screening, referral, and appointment process and, if they attended a mental health visit, steps implemented or supports utilized to attend. For patients who did not attend, we will similarly ask them to recount any steps they attempted to take and barriers they encountered.

Provider behavior (observations)

Study staff will spend one full week observing intake coordinators in the centralized mental health assessment, triage, and referral management center (i.e., PIC Resource Center). Observations will include detailed field notes of processes and procedures to understand leverage points for increasing engagement.

Analyses

Qualitative analysis

We will upload interviews and field notes into NVivo for data management and analysis. Analysis will be guided by an integrated approach that includes identification of a priori attributes of interest (i.e., EAST principles) and modified grounded theory, which provides a rigorous, systematic approach to identifying emergent codes and themes [47]. This approach uses an inductive process of iterative coding to identify recurrent themes, categories, and relationships. After initial exploration of data, a comprehensive coding scheme will be developed and applied to all data. The product will be a fine-grained descriptive analysis of the challenges to engagement in mental health services, including issues related to attendance at first visit following referral. Two members of the research team will separately double code a sample of transcripts to assess reliability of the coding scheme. Disagreements will be resolved through team discussion.

Quantitative analysis

To explore potential mechanisms, we will use descriptive statistics to examine the distribution of self-report (CFC and MSPSS) and behavioral task (KDDT) scores. We will use one-way ANOVAs to examine the relationships between attendance and scores on the CFC, MSPSS, and KDDT. These analyses are exploratory to inform future trials; the study will not be powered to detect effects.

Mixed methods analysis

We will integrate quantitative data from Aim 1 and interviews using the following taxonomy: the structure is Quan—QUAL, the function is to expand upon the quantitative findings to understand the process of engagement as experienced by stakeholders, and the process is connecting [47]. To integrate the quantitative and qualitative methods, we will follow the NIH guidelines for best practices [48]. We will use the quantitative data to identify patterns in the qualitative data. We will enter quantitative findings into NVivo as attributes of each participant (i.e., did/did not attend; referral to PIC or community; KDDT, CFC, and MSPSS scores). These attributes will be used to categorize and compare important themes among subgroups.

Aim 3 (multi-aim, non-randomized feasibility trial)

Aim 3 is rapidly prototype and test strategies to optimize engagement by using a series of rigorous, iterative tests.

In Aim 3, we will develop preliminary strategies to support attendance at initial mental health visits and iteratively test and refine these strategies through rapid prototyping. Secondarily, we will explore the role of two potential mechanisms, self-regulation and social support, in the use of engagement strategies and attendance at initial mental health visits.

Sample

Each strategy will be tested with approximately 5 patients. All patients must be (1) 18 years and older and able to communicate in English and (2) present with elevated suicidal ideation on item 9 of the PHQ-9 (item score ≥1) completed during a primary care visit. In addition, patients must not (1) currently be experiencing a psychotic episode requiring emergency services and/or precluding their ability to provide informed consent, (2) have a documented diagnosis of dementia in the past 2 years, (3) have primary care provider notes demonstrating that participation is not indicated, and (4) have already received a study engagement strategy following a different primary care visit. We will leverage the infrastructure of the Resource Center to recruit from the pool of patients who were referred for a mental health intake following a positive screen for depression or suicide risk in primary care based on the EHR. This provides a mechanism for recruiting patients while also ensuring that there is appropriate clinical screening and expertise to triage individuals in need of acute services. We anticipate that 25 patients will complete the temporal discounting and social support behavioral task and self-report measures, though additional patients may receive the strategies. Attendance data will be available from any individual who receives a strategy as part of routine care quality assurance.

Procedure

A team of experts in implementation science (n = 2), behavioral economics (n = 2), suicide (n= 2), and primary care and behavioral health (n = 4, total N = 10) will convene a half-day retreat to guide Aim 3 activities and develop strategies for rapid prototyping. This team will review key barriers and facilitators to attendance at initial mental health visits identified in Aims 1 and 2 and then generate strategies for facilitating treatment initiation that address barriers and leverage facilitators. The EAST framework will guide the selection of strategies [24]. We will also favor simple, established strategies [49] with freely available resources (e.g., Caring Contacts templates from Now Matters Now) to promote scalability [50], but will also consider other potential implementation strategies as indicated [31]. We will explore both low- and high-tech (e.g., EHR integration) strategies to maximize relevance across contexts. Based on the literature, previous work, and our conceptual model, we hypothesize that strategies that counteract delayed discounting tendencies (e.g., incentives) and foster emotional social support (e.g., Caring Contacts) will be needed [27].

We will then use rapid prototyping to test and refine the strategies identified by the team of experts [31], a process that has been applied successfully at Penn over the past 6 years [51,52,53,54]. The goal of rapid prototyping is to determine the best strategies for broader implementation in a subsequent trial. Prototype strategy designs will be validated through rapid-cycle tests in this real-world context and in a manner that is quick and inexpensive (see Table 2 for examples). This will allow us to fail fast and learn quickly. As we deploy strategies and learn from our experiences, we will adapt and iterate to refine promising strategies.

Table 2 Implementation strategy crosswalk
Full size table

Strategies will be rolled out in the PIC Resource Center over 6 months. During this period, strategies will be implemented by an embedded mental health intake coordinator as part of routine care. Consent to receive strategies is, therefore, not required. However, because participants will complete measures assessing acceptability and feasibility of these strategies, as well as temporal discounting and social support, we will employ an IRB-approved modified consent procedure prior to implementing strategies that includes a waiver of written consent. The modified consent procedure will consist of (1) a brief description of the study purpose, procedures, and potential risks; (2) notification that the strategy is part of a research project for which participation is voluntary; and (3) instructions on declining further participation. This modified approach will allow us to circumvent potential confounds associated with limiting a trial examining engagement to those who provide active, written informed consent to receive engagement strategies. By waiving the requirement of traditional, written informed consent, we will minimize the likelihood of a biased sample and maximize the validity and generalizability of our results [57, 58]. We anticipate that each strategy will be tested with ≥5 patients, but this is flexible; additional tests will be implemented as needed.

Measures

After testing each strategy, we will quantitatively assess patients’ perceptions of its feasibility, acceptability, and appropriateness using three brief, validated questionnaires: Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM), that were developed to better evaluate and monitor implementation outcomes [59]. Each questionnaire consists of four items (total of 12 items) rated on a 1–5 Likert-type scales ranging from “Completely disagree” to “Completely Agree.” The total score ranges from 4 to 20 for each measure, with higher scores indicating greater acceptability, appropriateness, or feasibility, respectively. Reliability was demonstrated with Cronbach’s alphas of .85 for acceptability, .91 for appropriateness, and .89 for feasibility, and test-retest reliability was demonstrated with correlations of .80 for acceptability, .73 for appropriateness, and .88 for feasibility.

Patients will then complete the temporal discounting and social support behavioral task and self-report measures (the KDDT, CFC, and MPSS) described in Aim 2.

Analyses

Descriptive statistics will be used to describe feasibility, acceptability, and appropriateness of strategies as assessed with the AIM, IAM, and FIM. We will also use logistic regressions to examine the impact of engagement strategies on attendance at initial mental health visits using attendance information derived from the administrative data (did/did not attend). These analyses will provide us with a preliminary estimate of the effectiveness of each engagement strategy. Finally, we will use descriptive analyses to examine scores on the temporal discounting and social support self-report and behavioral tasks, as well as relationships between scores, engagement strategies, and attendance at initial mental health visits. These analyses are exploratory to inform mechanisms to investigate in future trials.

Power analyses

Consistent with recommendations for pilot studies, we have not conducted power analyses for Aims 1–3. We instead focus on exploring the data at this stage [60], as well as examining the feasibility and acceptability of strategies to support attendance at initial mental health visits. Analyses will be oriented towards identifying promising strategies and mechanisms to focus on in subsequent studies.

Discussion

Primary care has been identified as a crucial setting for connecting individuals at risk for suicide to follow-up care, yet only half of patients referred to mental health treatment by primary care providers attend an initial visit [15, 16]. Surprisingly, no studies have yet developed or tested strategies to increase mental health treatment initiation for individuals at risk for suicide identified in primary care. This study aims to use frameworks and methods from behavioral economics, implementation science, and experimental therapeutics to develop acceptable, feasible, low-cost, and effective strategies to increase treatment initiation in this population. To do this, we will conduct a multiphase, mixed-methods study to identify characteristics of patients at risk for suicide who do and do not initiate mental health treatment, examine barriers and facilitators to treatment initiation among patients at risk for suicide, and rapidly prototype and test strategies to increase engagement.

This study has multiple strengths. First, we will partner with a large, diverse health system to identify barriers and facilitators to treatment initiation and develop and test potential solutions. Second, we will leverage rich administrative data to capture clinical characteristics and mental healthcare utilized both inside and outside the health system. This approach addresses key limitations of medical record use, which does not capture utilization outside the health system, and insurance claims, which lack verified diagnoses [61]. Third, unlike most studies of patient engagement, we will both quantitatively and qualitatively examine engagement strategies. Fourth, we will concurrently assess the perspectives of multiple stakeholder groups, including providers, patients, and health system leaders, which is a valuable approach that is rarely taken. Fifth, we will integrate methods and measures from behavioral economics, implementation science, and experimental therapeutics to develop and refine implementation strategies and elucidate generalizable mechanisms of engagement. Finally, we will rapidly prototype and test engagement strategies, which increases the potential for successful implementation and decreases the research-to-practice timeline.

Despite these strengths, the current study also has several limitations. In particular, the study will be conducted in a single, well-resourced health system with an existing integrated care program; the extent to which the results will generalize to other health systems is unknown. However, the demographic and economic diversity of the Penn Medicine patients may offset threats to external validity. In addition, given our focus on establishing feasibility and acceptability of implementation strategies, our sample size for testing strategies will be small. As such, statistical power will be limited, precluding our ability to draw firm conclusions about the strategies’ effectiveness. Furthermore, although we will examine associations between mechanisms of behavior change (i.e., self-regulation and social support) and treatment initiation among individuals at risk for suicide, our small sample size will preclude us from testing whether these mechanisms mediate the relationship between engagement strategies and treatment initiation. The current research, however, will be pivotal in paving the way for larger, confirmatory trials testing the effectiveness of these strategies and relevance of these mechanisms of behavior change. Finally, the current study is focused on treatment initiation rather than ongoing treatment engagement. Given that this is a formative trial, we chose to operationalize treatment engagement in this way to maximize parsimony, feasibility, and generalizability of our findings. It is possible, however, that effective strategies may differ for initial and ongoing treatment engagement. Examining strategies to improve ongoing treatment engagement is beyond the scope of the current trial, but we aim to broaden our operationalization of engagement in a subsequent larger, adequately powered trial.

In sum, this study will produce novel information about factors that enable and prevent primary care patients at risk for suicide from following up on mental health treatment. It will also produce a menu of strategies likely to increase treatment initiation among these individuals, along with potential mechanisms that may account for changes in treatment initiation. We anticipate that this project will lead to future studies testing the strategies developed to increase treatment initiation in a randomized, multi-site implementation trial. Our ultimate goal is to improve access to and engagement in mental health services for individuals at risk for suicide. This goal has the potential to reduce mortality among the majority of individuals with suicidal ideation who regularly attend primary care but not specialty mental health care.

Availability of data and materials

NA.

Abbreviations

AIM:

Acceptability of Intervention Measure

ALACRITY:

Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness

ANOVA:

Analysis of variance

CFC:

Consideration of Future Consequences scale

EAST:

Easy, Attractive, Social, and Timely

EBPs:

Evidence-based practices

EHR:

Electronic health records

FIM:

Feasibility of Intervention Measure

GAD-7:

General Anxiety Disorder-7

IAM:

Intervention Appropriateness Measure

KDDT:

Kirby Delay-Discounting Task

MSPSS:

Multidimensional Scale of Perceived Social Support

NIH:

National Institutes of Health

PHQ-9:

Patient Health Questionnaire-9

PIC:

Penn Integrate Care Program

RDoC:

Research Domain Criteria Project

SOBC:

Science of Behavior Change

References

  1. Centers for Disease Control and Prevention, National Center for Health Statistics. National Ambulatory Medical Care Survey (NAMCS) 2014; https://www.cdc.gov/nchs/ahcd/index.htm

  2. American Association of Suicidology. U.S.A. Suicide 2017: official final data. 2018. www.suicidology.org. Accessed 10 Dec 2018.

  3. Shepard DS, Gurewich D, Lwin AK, Reed GA Jr, Silverman MM. Suicide and suicidal attempts in the United States: costs and policy implications. Suicide Life Threat Behav. 2016;46(3):352–62.

    Article  PubMed  Google Scholar 

  4. Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS) [online]. 2005; http://www.cdc.gov/ncipc/wisqars.

  5. Alonzo D, Moravec C, Kaufman B. Individuals at risk for suicide: mental health clinicians’ perspectives on barriers to and facilitators of treatment engagement. Crisis. 2017;38(3):158–67.

    Article  PubMed  Google Scholar 

  6. Dixon LB, Holoshitz Y, Nossel I. Treatment engagement of individuals experiencing mental illness: review and update. World Psych. 2016;15(1):13–20.

    Article  Google Scholar 

  7. Erlangsen A, Lind BD, Stuart EA, Qin P, Stenager E, Larsen KJ, et al. Short-term and long-term effects of psychosocial therapy for people after deliberate self-harm: a register-based, nationwide multicentre study using propensity score matching. Lancet Psychiatry. 2015;2(1):49–58.

    Article  PubMed  Google Scholar 

  8. Miller IW, Camargo CA, Arias SA, Sullivan AF, Allen MH, Goldstein AB, et al. Suicide prevention in an emergency department population: the ED-SAFE study. JAMA Psychiatry. 2017;74(6):563–70.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Doupnik SK, Rudd B, Schmutte T, Worsley D, Bowden CF, McCarthy E, et al. Association of suicide prevention interventions with subsequent suicide attempts, linkage to follow-up care, and depression symptoms for acute care settings: a systematic review and meta-analysis. JAMA Psychiatry. 2020;77(10):1021–30.

    Article  PubMed  Google Scholar 

  10. Kessler R. Mental health care treatment initiation when mental health services are incorporated into primary care practice. J Am Board Fam Med. 2012;25(2):255–9.

    Article  PubMed  Google Scholar 

  11. Younes N, Passerieux C, Hardy-Bayle MC, Falissard B, Gasquet I. Long term GP opinions and involvement after a consultation-liaison intervention for mental health problems. BMC Fam Pract. 2008;9:41.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Cunningham PJ. Beyond parity: primary care physicians’ perspectives on access to mental health care. Health Aff. 2009;28:w490–501.

    Article  Google Scholar 

  13. Blackwell D, Lucas J, Clarke T. Summary Health Statistics for U. S. adults: National Health Interview Survey, 2012. Atlanta, GA: Vital Health Stat National Center for Health Statistics; 2014.

  14. Ahmedani BK, Simon GE, Stewart C, Beck A, Waitzfelder BE, Rossom R, et al. Health care contacts in the year before suicide death. J Gen Intern Med. 2014;29(6):870–7.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Solberg LI, Crain AL, Jaeckels N, Ohnsorg KA, Margolis KL, Beck A, et al. The DIAMOND initiative: implementing collaborative care for depression in 75 primary care clinics. Implement Sci. 2013;8(1):135.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Whitebird RR, Solberg LI, Jaeckels NA, Pietruszewski PB, Hadzic S, Unützer J, et al. Effective implementation of collaborative care for depression: what is needed? Am J Manag Care. 2014;20(9):699–707.

    PubMed  PubMed Central  Google Scholar 

  17. Oldham M, Kellett S, Miles E, Sheeran P. Interventions to increase attendance at psychotherapy: a meta-analysis of randomized controlled trials. J Consult Clin Psychol. 2012;80(5):928–39.

    Article  PubMed  Google Scholar 

  18. Lowenstein G, Asch DA, Volpp KG. Behavioral economics holds potential to deliver better results for patients, insurers, and employers. Health Aff. 2013;32(7):1244–50.

    Article  Google Scholar 

  19. Volpp KG, Loewenstein G, Asch DA. Behavioral economics and health. In: Glanz K, Rimer BK, Viswanath V, editors. Health behavior: theory, research, and practice. San Francisco: Jossey-Bass, Inc./Wiley; 2015.

    Google Scholar 

  20. Keller PA, Harlam B, Loewenstein G, Volpp KG. Enhanced active choice: a new method to motivate behavior change. J Consum Psychol. 2011;21(4):376–83.

    Article  Google Scholar 

  21. Sunstein CR. Nudges that fail. Behavioural Public Policy. 2017;1(1):4–25.

    Article  Google Scholar 

  22. Wu SJ, Paluck EL. Designing nudges for the context: golden coin decals nudge workplace behavior in China. Organ Behav Hum Decis Process. in press.

  23. Nielsen L, Riddle M, King JW, Aklin WM, Chen W, Clark D, et al. The NIH science of behavior change program: transforming the science through a focus on mechanisms of change. Behav Res Ther. 2018;101:3–11.

    Article  PubMed  Google Scholar 

  24. Service O, Hallsworth M, Halpern D, Algate F, Gallagher R, Nguyen S, et al. EAST: four simple ways to apply behavioral insights; 2014.

    Google Scholar 

  25. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol. 2007;3:1–27.

    Article  PubMed  Google Scholar 

  26. Post EP, Cruz M, Harman J. Incentive payments for attendance at appointments for depression among low-income African Americans. Psychiatr Serv. 2006;57(3):414–6.

    Article  PubMed  Google Scholar 

  27. Comtois KA, Kerbrat AH, DeCou CR, Atkins DC, Majeres JJ, Baker JC, et al. Effect of augmenting standard care for military personnel with brief caring text messages for suicide prevention: a randomized clinical trial. JAMA Psychiatry. 2019;76(5):474–83.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Reger MA, Gebhardt HM, Lee JM, Ammerman BA, Tucker RP, Matarazzo BB, et al. Veteran preferences for the caring contacts suicide prevention intervention. Suicide Life-Threat Behavior. 2019;49(5):1439–51.

  29. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–45.

    Article  PubMed  Google Scholar 

  30. Crossing the Quality Chasm. A new health system for the 21st century. Washington, DC: National Academy Press; 2001.

    Google Scholar 

  31. Dopp AR, Parisi KE, Munson SA, Lyon AR. Integrating implementation and user-centred design strategies to enhance the impact of health services: protocol from a concept mapping study. Health Res Policy Syst. 2019;17(1):1.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Beidas RS, Volpp KG, Buttenheim AN, Marcus SC, Olfson M, Pellecchia M, et al. Transforming mental health delivery through behavioral economics and implementation science: protocol for three exploratory projects. JMIR Res Protoc. 2019;8(2):e12121. https://doi.org/10.2196/12121.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Kirby KN, Marakovic NN. Delay-discounting probabilistic rewards: rates decrease as amounts increase. Psychon Bull Rev. 1996;3(1):100–4.

    Article  CAS  PubMed  Google Scholar 

  35. Kirby KN. One-year temporal stability of delay-discount rates. Psychon Bull Rev. 2009;16(3):457–62. https://doi.org/10.3758/PBR.16.3.457.

    Article  PubMed  Google Scholar 

  36. Strathman A, Gleicher F, Boninger DS, Edwards CS. The consideration of future consequences: weighing immediate and distant outcomes of behavior. J Pers Soc Psychol. 1994;66(4):742–52.

    Article  Google Scholar 

  37. Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. J Pers Assess. 1988;52(1):30–41.

    Article  Google Scholar 

  38. Fernandez ME, ten Hoor GA, van Lieshout S, et al. Implementation mapping: using intervention mapping to develop implementation strategies. Front Public Health. 2019;7(158)1–15.

  39. Altman M, Huang TTK, Breland JY. Design thinking in health care. Prev Chronic Dis. 2018;15:E117.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Wolk CB, Last BS, Livesey C, Oquendo MA, Press MJ, Mandell DS, et al. Addressing common challenges in the implementation of collaborative care for mental health: the Penn integrated care program. Ann Fam Med. 2021;19(2):148–56. https://doi.org/10.1370/afm.2651.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Press MJ, Howe R, Schoenbaum M, et al. Medicare payment for behavioral health integration. N Engl J Med. 2017;376(5):405–7.

    Article  PubMed  Google Scholar 

  42. Archer J, Bower P, Gilbody S, Lovell K, Richards D, Gask L, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev. 2012;10:Cd006525.

    PubMed  Google Scholar 

  43. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.

    Article  PubMed  Google Scholar 

  45. Odum AL. Delay discounting: I'm a k, you’re a k. J Exp Anal Behav. 2011;96(3):427–39.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Zimet GD, Powell SS, Farley GK, Werkman S, Berkoff KA. Psychometric characteristics of the multidimensional scale of perceived social support. J Pers Assess. 1990;55(3-4):610–7.

    Article  CAS  PubMed  Google Scholar 

  47. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4):1758–72.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Creswell J, Klassen A, Plano Clark V, Smith K. Best practices for mixed methods research in the health sciences. In: the Office of Behavioral and Social Sciences Research, ed. Washington, DC: National Institutes of Health; 2011.

    Book  Google Scholar 

  49. Mehta SJ. Scaling and spreading innovation in health care delivery. Jt Comm J Qual Patient Saf. 2018;44(10):564–5.

    PubMed  Google Scholar 

  50. Center for Health Care Innovation. 2019; https://healthcareinnovation.upenn.edu/. Accessed 28 Jan 2019.

  51. Thaler RH, Sustein CR. Nudge: improving decisions about health, wealth, and happiness. Revised and expanded edition. New York: Penguin Books; 2009.

    Google Scholar 

  52. Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, et al. Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res. 2014;16(2):e42. https://doi.org/10.2196/jmir.3173 Erratum in: J Med Internet Res. 2015;17(4):e69.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Asch DA, Terwiesch C, Mahoney KB, Rosin R. Insourcing health care innovation. N Engl J Med. 2014;370:1775.

    Article  CAS  PubMed  Google Scholar 

  54. Asch DA, Rosin R. Innovation as discipline, not fad. N Engl J Med. 2015;373(7):592–4.

    Article  CAS  PubMed  Google Scholar 

  55. Vlaev I, King D, Darzi A, Dolan P. Changing health behaviors using financial incentives: a review from behavioral economics. BMC Public Health. 2019;19(1):1059.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Schwebel FJ, Larimer ME. Using text message reminders in health care services: a narrative literature review. Internet Interv. 2018;13:82–104.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Zelen M. Randomized consent designs for clinical trials: an update. Stat Med. 1990;9(6):645–56.

    Article  CAS  PubMed  Google Scholar 

  58. Simon GE, Beck A, Rossom R, Richards J, Kirlin B, King D, et al. Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial. Trials. 2016;17(1):452.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45:626–9.

    Article  PubMed  Google Scholar 

  61. Heintzman J, Bailey SR, Hoopes MJ, Le T, Gold R, O’Malley JP, et al. Agreement of Medicaid claims and electronic health records for assessing preventive care quality among adults. J Am Med Inform Assoc. 2014;21(4):720–4.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors wish to thank their partners in the Penn Integrated Care program including Matthew Press, MD, MSc, and Eleanor Anderson, MD for their collaboration.

Funding

This study is funded by the National Institute of Mental Health (R21MH123851). This article was also supported by the Office of Academic Affiliations, Advanced Fellowship Program in Mental Illness Research and Treatment, Department of Veterans Affairs (Dr. Khazanov), and the VISN 4 Mental Illness Research Education and Clinical Center (Dr. Oslin). The funding source had no role in the design of the study and will not have any role in the conduct of the study, data analysis, interpretation of results, or the writing of study reports.

Author information

Authors and Affiliations

  1. Mental Illness Research, Education, and Clinical Center of the Veterans Integrated Service Network 4, Crescenz Veterans Affairs Medical Center, Philadelphia, PA, USA

    Gabriela Kattan Khazanov & David W. Oslin

  2. Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA

    Gabriela Kattan Khazanov, Shari Jager-Hyman, Molly Candon, Alison Buttenheim & Courtney Benjamin Wolk

  3. Department of Psychiatry, University of Pennsylvania School of Medicine, Philadelphia, PA, USA

    Shari Jager-Hyman, Molly Candon, Matteo F. Pieri, David W. Oslin & Courtney Benjamin Wolk

  4. Center for Health Incentives and Behavioral Economics, University of Pennsylvania School of Medicine, Philadelphia, PA, USA

    Joseph Harrison & Alison Buttenheim

  5. Philadelphia College of Osteopathic Medicine, School of Professional and Applied Psychology, Philadelphia, PA, USA

    Joseph Harrison

  6. Department of Family and Community Health, University of Pennsylvania School of Nursing, Philadelphia, PA, USA

    Alison Buttenheim

Authors
  1. Gabriela Kattan Khazanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shari Jager-Hyman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joseph Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Molly Candon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alison Buttenheim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matteo F. Pieri
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David W. Oslin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Courtney Benjamin Wolk
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CBW, SJH, MC, and AB designed the study. CBW and SJH developed the grant application upon which this manuscript is based. GKK conceived the presentation of the grant proposal as a study protocol manuscript and drafted the manuscript. All authors critically revised the manuscript for important intellectual content and gave final approval of the version submitted for publication.

Corresponding author

Correspondence to Gabriela Kattan Khazanov.

Ethics declarations

Ethics approval and consent to participate

This project, “Leveraging behavioral economics and implementation science to engage suicidal patients in mental health treatment”, was approved by the University of Pennsylvania IRB (protocol number 844601).

Consent for publication

NA.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Shari Jager-Hyman is the co-first author.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khazanov, G.K., Jager-Hyman, S., Harrison, J. et al. Leveraging behavioral economics and implementation science to engage patients at risk for suicide in mental health treatment: a pilot study protocol. Pilot Feasibility Stud 8, 181 (2022). https://doi.org/10.1186/s40814-022-01131-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40814-022-01131-y

Keywords

Pilot and Feasibility Studies

ISSN: 2055-5784