Reducing Suicidal Ideation among Turkish Migrants in the Netherlands and in the UK A Pilot RCT of a guided Online Intervention

Background: The evidence for the effectiveness of e-mental health interventions among ethnic minorities is still preliminary. This mixed methods study investigates the feasibility of a culturally adapted, guided online intervention with the intention to understand how it works and for whom to inform refinement. It also examines indications for its effectiveness in reducing suicidal ideation when compared with the treatment as usual. Methods: Turkish migrants (first and second generation) with mild to moderate suicidal thoughts were recruited from the general population using social media and newspaper advertisements. The intervention group obtained direct access to a 6 weeks guided online intervention while participants in the waiting list condition had to wait for 6 weeks. The intervention is based on an existing online intervention and was culturally adapted. Participants in both conditions completed baseline, post-test, and follow up questionnaires on suicidal ideation (primary outcome), depression, worrying, hopelessness, suicide attempt and self-harm, acculturation, quality of life, and usability. In addition, participants were interviewed to examine the feasibility and mechanisms of action in more depth. The responses were analysed by inductive thematic analysis. Results: A total of 85 people signed up via the study website and 18 (10 intervention, 8 waitlist control) were included. While the therapeutic benefits were emphasised (e.g. feeling connected with the intervention), the feasibility was judged to be low. The main reasons given were: not having severe suicidal thoughts and not being represented by the culturally adapted intervention. No suicide attempts were recorded during the study. The suicidal ideation, depression and hopelessness scores were improved in both groups. Conclusion: The online intervention, as might be expected in a small feasibility study, was not superior to the control condition. Future studies need to attend the implementation issues raised including measures of stigma, acculturation, careful cultural adaptations alongside more attention to coaching and relational support. They should also consider how to improve engagement alongside selection of those who are motivated to use online interventions and offer alternatives for those who are not.


Background
Suicide is a global public health problem with enormous consequences at individual and societal levels (1). The international lifetime prevalence of suicidal ideation, plans and attempts in general population is 9.2%, 3.1% and 2.7%, respectively (2). Members of some ethnic groups are at higher risk for suicidal behaviours compared to others (3). In Europe, Turkish populations are among the largest ethnic minority populations and they have disproportionate rates of suicidal behaviours compared to the ethnic majorities in their respective host countries (4). In the Netherlands, there is an elevated risk of suicidal ideation in Turkish adolescents (38.1%) when compared with ethnic Dutch (17.9%) adolescents (5). Suicide attempt risk is increased 2-5 fold among Turkish migrant women aged 14 and 25 when compared with the same aged and locally born women in the Netherlands (6), Germany (7) and Switzerland (8). The causes of the increased prevalence rates are not clear yet.
Some have argued that this might be associated with gender related factors such as domestic violence and honour related violence (e.g. being forced into an unwanted marriage), which are important life events among women who are presenting suicidal behaviours worldwide (9)(10). Others have stressed that people might encounter difficulties in their adaptation process to host countries, and interpersonal and structural discrimination within institutions, contributing to the elevated risk of suicidality (11)(12).
Existing guidelines for treating suicidal behaviours in the UK and in the Netherlands recommend cognitive behavioural therapy (CBT) and CBT based interventions, such as Dialectical Behavioural Therapy (DBT), in managing suicidal behaviours which are delivered face-to-face (13)(14). These interventions are based on a general cognitive model which is expanded to suicidal behaviours (15)(16). How one interprets a situation determines emotional and behavioural reactions to those situations (15)(16). Interpretation is distorted by errors in thinking such as overgeneralization and emotional reasoning. CBT interventions aim to restructure thinking errors with the intention to help individuals to make sense of situations from a more realistic point of view (15). The efficacy of the Mentalization-Based Treatment (MBT) has also been demonstrated in several RCTs (17)(18) and longitudinal studies (19) among adult and adolescent populations. Mentalization is the capacity to understand actions in terms of thoughts and feelings (18). When mentalizing is compromised in interpersonal relationships, negative thoughts are experienced in greater intensity leading to an urgent need for distraction. In this context, suicidal behaviours may serve as distraction (18).
Cultural barriers such as stigma and shame attached to suicide might prevent them from utilizing these interventions (21)(22). Additionally, poor language proficiency of the help-seeker and cultural mismatch between the mental health professional and the help-seeker often result in communication barriers during the help-seeking process (23). It is therefore a global challenge to optimize these interventions for migrants and ethnic minorities (24).
In recent years, online interventions have been introduced into the mental health services as an addition, or alternative for, face-to-face service delivery. These interventions can be delivered through PC's, mobile phones or tablets and can be either guided or un-guided. Guidance can be delivered by a clinician or a trained coach (25) and is aimed at motivating patients, to explain things which are unclear, and to provide feedback on the content of the sessions (25). Unguided e-mental health interventions may involve automated feedback but do not provide any professional support related to the therapeutic content (25). Guided interventions are often more effective than unguided interventions (26) but unguided interventions are more scalable and can reach larger groups of people (27).
There is growing evidence supporting the value of online interventions in engaging with people from across the life span and from different ethnic backgrounds in treatment of common mental disorders such as depression and anxiety (27). There are also several randomized controlled trials testing the effectiveness of online interventions for people with suicidal ideation (28- [29][30]. Recently, metaanalytic studies indicated significant beneficial treatment effect of those interventions compared to treatment as usual (g = -0.26; 95% CI: -0.48, -0.03 (31). These studies however, have been carried out in general populations. Their effectiveness for even more vulnerable populations for instance, migrants and ethnic minorities with suicidal ideation, is lacking (32)(33)(34)(35).
In this study we have used an e-mental health intervention for suicidal ideation, which was developed for the general population in the Netherlands by van Spijker and colleagues (36). We have adapted this intervention for the Turkish migrants in the Netherlands and in the UK (21)(22)(23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36)(37). Cultural adaptation is defined as 'the systematic modification of an evidence-based treatment or intervention protocol to consider language, culture, and context in such a way that it is compatible with the individual's cultural patterns, meanings, and values' (38). CBT interventions offer flexibility to be adapted according to the needs and expectations of a diverse help-seeker population in multicultural health care (24). Culturally adapted online interventions seem promising for ethnic minorities in general (39) and for Turkish migrants specifically (40)(41).
The objectives of this pilot study are twofold: 1) to investigate the feasibility of the adapted online intervention among Turkish migrants in the UK and in the Netherlands; 2) and to investigate whether there is an indication of the effectiveness of the culturally adapted online intervention in reducing suicidal ideation when compared with the treatment as usual.

Design, Settings and Participants
This study is a randomized controlled trial in which patients were randomized to the guided online intervention or to a wait-list control group. The control group could directly access a website with psycho-education about the reasons for suicidal thoughts, risk factors and where to seek help (e.g. 113 online, Samaritans, mental health organisations) and could start with the intervention after the post-test measurement at 6 weeks. As part of the feasibility, semi-structured telephone interviews were held at 6 weeks with the intention to obtain in-depth information about how the intervention is implemented and the contextual factors related with the uptake of the intervention in daily life.
Participants were recruited between January 2017 and October 2018.
Recruitment took place among the Turkish-speaking population (i.e. people of Turkish, Kurdish and Turkish Cypriot background;  in the Netherlands and in the UK. Participants were recruited from the general population through newspaper advertisements, TV programmes, banners on relevant websites (e.g. of community and health organizations), social media and public events.
These events were organised in collaboration with the community organisations representing the population of interest.
Eligible participants were 18 years and older, and had Turkish background (being born in Turkey or having at least one parent being born in Turkey) had a suicidal ideation score of 1 or higher on the Beck Scale for Suicide Ideation (BSS), had access to a PC and internet, and consented to provide their name, telephone number, e-mail address and the e-mail of their GP. These data were needed for our safety procedure. Already receiving help, regardless of the source, was not an exclusion criterion.
A study website (http://kiymacanina.org/) was created where potential participants could find more information about the study and register in Turkish, English and Dutch. Those who registered received an e-mail with further information, an informed consent form and a link to the baseline questionnaire.
Those who returned the informed consent, provided their contact details and those of their GPs`, filled out the baseline questionnaire, and did not fulfil our exclusion criteria, were included. The randomisation scheme was derived using random allocation software by an independent researcher.
Randomisation was stratified for the UK and the Netherlands and took place in a 1:1 ratio. The outcome was communicated to the participant by e-mail with either a log-in code for the intervention or a link to a website with general information on suicidality for the wait-list control group. The study protocol is described in more detail elsewhere (37) and was registered in the Netherlands Trial Register NTR5028 (see https://www.trialregister.nl/trial/4926).

Ethics Statement
This study was approved by the Medical Ethics Committee of the VU University Medical Centre in the Netherlands (registration number 2014. 187) and by the Queen Mary University of London Research Ethics Committee in the UK (registration number QMERC2014/46). Written informed consent of participants was obtained after the study and all procedures had been fully explained in writing.
Participants could ask questions by e-mail or telephone if wanted.

Safety
As this study involved vulnerable people who are at risk of suicide, a safety protocol was used (37). In summary: every participant (in both conditions) was asked to fill out the BSS once in two weeks throughout their participation. Whenever a participant exceeded the cut-off score of 29 on the BSS, we phoned the participant to perform a risk assessment. If necessary, we contacted their GP. The phone calls were made by a psychologist in the research team under the supervision of a licensed clinical psychologist in the Netherlands and a licenced consultant psychiatrist in the UK who were both experienced in suicide prevention. Participants' GPs were also contacted if they could not be reached.

Intervention
The original version of the intervention is developed by van Spijker and colleagues (43). RCTs into the unguided version of this intervention showed its effectiveness in reducing suicidal ideation compared to the treatment as usual in general Dutch (d=0.2 ; 27) (28) an Belgian populations (d= 0.34) (30), but it showed no effect when implemented in an Australian population (28).
The intervention is based on the CBT framework (44). Within this framework, some mindfulness exercises were also included. The main principle is that, worry, rumination and repetitive suicidal ideation each produce obsessive attention to particular thoughts, sometimes resulting in a desire to end consciousness as a way to end the tantalizing repetition of suicidal thoughts (43). Thus, the aim of this intervention is to enhance controlled thinking (i.e. focusing on postponing worrisome thoughts to specific time-slots "worry times'' of the day and not thinking of these thoughts for the rest of the day).
The intervention consists of six modules: 1) the repetitive character of suicidal thoughts, 2) regulating intense emotions, 3) identifying automatic thoughts; 4) thinking patterns, 5) thought challenging, and 6) relapse prevention (37). Each module contains a theory section, a weekly assignment and several exercises. Weekly assignment is an essential element of the intervention encouraging participants to practice the techniques they learn from the intervention in daily life. For example, the first module explains that suicidal thoughts can develop out of self-protection, as keeping on living may seem worse than dying. Similarities between worry and suicidal thinking are also outlined. A weekly assignment involves tallying suicide-related thoughts to obtain an idea of how often these thoughts occur, while the exercises are aimed at learning to manage these repetitions better by introducing worry postponement. Participants are advised to do one module per week.
Guidance was available from coaches (2 in the UK and 2 in the Netherlands), who were supervised by a team of three experts including a psychologist, a licenced clinical psychologist (NL) and a consultant psychiatrist (UK). The coaches were bi-lingual (English-Turkish or Dutch-Turkish) and were either students at a Masters level or practitioners seeing patients. They received training about how the online intervention works, safety protocol, referral system in both countries and their roles and responsibilities whilst providing guidance to the participants. Regular supervision meetings through skype and/or e-mail communications were arranged.
Coaches provided online feedback to participants after their completion of each module. The aim of this personalised feedback was to help participants to understand the exercises and homework assignments as explained in the lessons. Moreover, it was used to motivate the participants to continue with the intervention.

Cultural adaptation
We adapted this intervention linguistically and culturally for the Turkish migrant populations in line with the evidence indicating that interventions work better when adapted to local settings (39). The decisions made during the cultural adaptation process are outlined in appendix B. In summary, we used the ecological validity model of Bernal and colleagues (38). This framework retains the core principles of the CBT in order to preserve treatment validity but permits flexibility (45). The model delineates eight dimensions when culturally adapting an intervention: the use of appropriate language, persons (cultural similarities/differences between the client and clinician which shape the therapeutic relationship), metaphors (symbols and concepts), content (cultural knowledge), concepts (treatment concepts that are culturally congruent), goals (that support adaptive cultural values), methods (cultural enhancement of treatment methods) and context (consideration of acculturation, social context) (see Appendix B).  (18) for instance, a well-known MBT based exercise called "safe place" was incorporated into the crisis plan in the intervention. This exercise uses guided imaginary and encourages people to create an imaginary safe place that they could visit whenever they feel the need to be grounded (18) (see appendix B).

Deviations from the study protocol
The sample size of this RCT was calculated based on the expected effect of the primary outcome measure: the reduction in frequency and intensity of suicidal thoughts as measured with the BSS (d=0.40). The rationale for this decision was based on the trial of van Spijker who found an effect size of d=0.2 for an unguided version of the treatment (28). We expected a higher effect size since we provided personal coaching which generally leads to higher effect sizes (26). Based on power of 0.80 and an alpha of 0.05, 100 participants were needed in each condition. Given the expected drop-out of 30%, the total sample size was determined as being 286 (37).
Although intended to be a definitive trial, it did not succeed and at best became a feasibility study with process evaluation to understand the components and how they operate. We conducted in depth interviews with the participants who completed the intervention and consented to be interviewed on feasibility issues. Furthermore, we have excluded the following questionnaires from the study: Suicidal Ideation Attribution Scale, the Hospital Anxiety and Depression Scale and an item measuring the satisfaction with the treatment. This was done to reduce the burden on participants and to increase their motivation to fill out our questionnaires. Moreover, we did not include three months follow-up assessments (T4) in the analyses and have done completers analyses. The reason was the underpowered study design which did not allow us to perform complex statistical procedures such as multiple imputation. We have used T4 to monitor the safety of the participants only. Finally, in the original protocol we described that we would add two additional modules on self-harm to the original intervention. In the end we decided to keep the focus on suicidal thoughts only (37).

Outcome Measures
As part of the feasibility, the system usability scale was used and interviews were conducted. The feasibility was measured at post-test (6 weeks after baseline: T3). Usability of the online intervention was measured with the System Usability Scale (SUS). The SUS is composed of 10 statements that are scored on a 5-point scale of extent of agreement (score 0 to 100). The reliability is good (Cronbach's alpha 0.91) (46). Interventions with scores of 70 and above are accepted as highly usable (46) and scores between 50 and 70 indicate acceptable usability of an intervention.
Interventions with scores 50 and below are subject to concerns about their usability by the target population and should be investigated further (46).
In order to answer the research question about the indications of effectiveness, online questionnaires were used. The primary outcome measure in this study is the reduction in the frequency and intensity of suicidal thoughts. This was measured with the BSS (47) at baseline (T0), 2 and 4 weeks into the intervention (T1 and T2), and at (T3). The BSS is a 21-item measure assessing the severity of the suicidal ideation (48)(49). Each item is scored from 0 to 2. The total score is obtained by adding the first 19 items and ranges from 0 to 38. High score represents high suicidal ideation. The BSS has good psychometric properties in English (47)(48)(49) and in Turkish (50)(51).
Secondary outcome measures are measured at T0 and at T3 and included the following: depression, hopelessness, worrying, quality of life, self-harm behaviour and suicide attempt, acculturation. Each item is scored from 0 to 3. The severity ranges from minimal depressed (score lower than 13) to severely depressed (scores between 29 and 63 (52). It is a reliable and valid (52) measure for assessing depression. The BDI has been validated in Turkish and Dutch population (53).
Hopelessness was measured with the Beck Hopelessness Scale (BHS), and contains 20 true and false statements (54). Each statement is scored from 0 to 1 and the total score ranges from 0 to 20. A high score indicates high degree of hopelessness. The instrument has good psychometric properties (55). This is also true for the Turkish version (56).
Worrying The Penn State Worry Questionnaire (PSWQ-PW) is a 15 item inventory assessing the weekly status of pathological worry (57). Each item is scored on a 7 point rating scale, ranging from never 0 (never) to 6 (almost always). The total score ranges from 0 to 90 with a high score indicating more worrying. PSWQ-PW shows good reliability and convergent validity (57). The Turkish version demonstrated reliability (58).

Quality of life The Euro Quol (EQ-5D) is an instrument measuring health quality of life and has 5
items: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression (59). Each item is required to be rated as 1 (no problem), 2 (some problem) or 3 (extreme problem). The current health state is also rated on a scale ranging from 0 (worst imaginable state) to 100 (best imaginable state).

Both Dutch and Turkish versions have been validated (60-61).
Suicide Attempt and Self-Harm (SASH) Four questions measuring the previous suicide attempt and the presence of self-harm were taken from the original Self-Harm Questionnaire (62).

Acculturation was measured with an adapted version of the Lowlands Acculturation Scale (LAS)
representing the difficulties that migrants might face. On a 6 point rating scale, item scores range from 1 (not applicable) to 6 (very applicable). The instrument is validated among Turkish migrants living in the Netherlands (63). It has been adapted to measure the 2 dimensions of acculturation: participation (4 items) and maintenance (11 items (64-65). The subscale participation measures tendency to participate in social life of the host country such as interacting with other minority and majority groups (score range 4 to 23). The subscale maintenance measures the tendency to maintain one`s culture of origin such as preferring to interact with people from same ethnic background (score range 11 to 60). Higher scores indicate a greater degree of participation and maintenance. The new sub scales were reliable (chronbach's alpha 0.86 for both scales) and internally consistent (participation α = 0.79; maintenance α = 0.80 (66).

Analyses
We defined feasibility as engagement with the intervention (31), its usability in daily life, as well as the potential for delivering a full trial in the future. The following components were assessed during the interviews in order to identify the facilitators and barriers influencing the engagement with the intervention. Cultural relevance (i.e. familiarity and relevance of the therapeutic content to one`s cultural background), cultural appropriateness (i.e. appropriateness of the therapeutic content in terms of the cultural context) and acceptance (i.e. feedback on the experience of using the intervention in real life (27).
The interviews were conducted with the participants who completed the intervention and consented to be interviewed (N=12) and started with an open question: 'What was your overall experience during your participation?' A topic guide was used for the remainder of the interview which was created on the basis of the relevant literature and discussions with the rest of the researchers taking part in the study (see Appendix A for the topic guide). The interviews were approximately 30 minutes long and were recorded for verbatim transcription.
We used thematic analysis as a means of identifying, analysing and reporting explanatory models, and for understanding which elements of the intervention facilitated or hindered participantsp rogress during their participation and how the intervention can be optimised to increase its acceptability, relevance and user-friendliness (67). Pseudonyms were assigned to each interviewee.
First, the code system (and categories and themes developed on the basis of the coding process) was developed gradually and collaboratively. The code system was developed on theoretical grounds and included following categories: Definition of facilitators and barriers, specific examples for cultural relevance, appropriateness and acceptability of the intervention and specific recommendations for further improvement. Each of these categories had a number of sub-categories and codes. The definition of therapeutic gains as a result of using the intervention (e.g. therapeutic alliance) emerged during the analysis. This category was considered as important and decided to be analysed separately. The coding system was developed by the first author and was checked independently by a second person (Y.A.) who was not involved in the research. Once an agreement was reached, they were further developed, refined and applied to the transcripts. The first author was the main coder and Y.A. involved as a second coder, who systematically counter-checked the coding, to assure the robustness and the internal validity. The data was coded manually. Disagreements over the coding were discussed between the main and the second coder and where necessary experts (S.R.) were consulted. Detailed descriptive accounts were produced for each major theme alongside the related extracts from participants' transcripts. Analysis continued until no new themes emerged from the transcripts (see table 4 for the themes).
The RCT was carried out in accordance with the CONSORT guidelines (see consort checklist).
First, t-tests and chi-square tests, as implemented in SPSS, were used to compare the baseline characteristics of those who were allocated in the intervention group with those who were allocated to the waitlist control group. Second, we tested for indications for effectiveness. We used Bayesian Repeated Measured ANOVA as implemented in the JASP (version 0.9.2), which is a free and opensource graphical programme for statistical analysis. The rationale for choosing Bayesian approach over the classical inferential approach is based on the following advantages: Bayesian approach considers all possible models (null model, between and within group differences) and assigns more weight to those models that predict data relatively well (68)(69). Classical inferential model selects the best model, estimates its parameters and might produce overconfident conclusions on data by neglecting model uncertainty (68)(69). Since the present study design is underpowered, we assumed that Bayesian approach would account for the uncertainty of the all possible models and would allow us to make more reliable conclusions based on the existing pilot data (69). Additionally, we did a sensitivity analysis on those participants who reported severe suicidal ideation at baseline to see if the indications for an effect of the intervention were stronger than in the whole group including those with very mild suicidal ideation.

Safety
We did monitor all the participants carefully through guidance and assessments (t1 and t2: bi-weekly assessments; t3: post-test; t4: 3 months follow-up). We did not call any of the participants as none of the participants exceeded the cut-off score of 29 on the BSS at any time during their participation.
Thus, the safety protocol was never activated. There were no suicide attempts or suicides during their participation in the study.

Qualitative Findings
The thematic analysis on the 12 interviews identified 3 overarching themes (see table 3).

Therapeutic Alliance
All participants commented on developing therapeutic alliance (i.e. the relationship between coaches and the participants) as a result of using the intervention. Many participants identified personalised feedback as one of the key components helping them to benefit from the intervention. For some participants, personalised feedback did not only motivate them to continue but it also provided a safe environment to disclose their experiences: "Receiving feedback was like exchanging letters with someone….Sometimes you cannot talk to everyone about certain things. But receiving feedback and being able to respond to it, was like a relief …. As I went through them, I kept on discovering new things about myself" participant A.

Self-Management
Several participants were going through important life events for instance, domestic violence, loss of a loved one and work related stress which resulted in crises. Crisis was often defined as feeling confused about how to handle stress. The psycho-educational aspect of the exercises and the feedback helped them to understand these crisis situations: "I was going through a trauma….and was not able to make much sense of what was happening to me…the feedback helped me to make sense of it all. It helped me to explain things from a scientific "Sometimes when I was panicking about something, I was letting myself to worry…to think of the worst case scenario that could happen to me… I was thinking of that for 10 minutes or so and afterwards I was able to feel better…..Managing to do that was really helpful….." participant F " It [following the intervention] gave me some peace of mind as I was doing something at least" Participant H

Feeling connected
Feeling connected with the intervention and/or with the personal coach appeared as a strong facilitator. Being able to relate to the content helped them feel connected with the intervention. Several participants identified cultural familiarity as a pleasant experience helping them to feel more connected with the intervention. Those who feel connected were also able to relate to the intervention (i.e. cultural relevance) and often found it appropriate (i.e. culturally appropriate): "The intervention was very familiar and it felt like I was not only getting professional help but was also talking to a friend" Participant E "I think all the examples were appropriate to the Turkish culture…They were also representative of the types of problems that migrant populations are likely to face" Participant G In terms of the acceptability of the intervention, participants expressed contradictory opinions. For some of them, the "self-help" principles made it difficult to use the intervention. They emphasised that guidance and cultural adaptations were not sufficient: "The difficulty with the online therapy is that, we need to do things on our own. When you see a psychologist….when there is a person in front of you….you feel more in control….I think there should be a psychologist in front of you and you should feel pushed….Do you see my point? After all not feeling in control is the main reason why we need psychological help….Isn`t it?" Participant H

Not feeling connected
Those who did not consider themselves as having severe suicidal ideation, did not feel connected with the intervention. They often reported feeling uncomfortable when they thought they were being considered as a severe case:

More diversity
Many participants recommended to include more diversity in the context (e.g. more case examples representing different backgrounds). All participants commented on the usefulness of the online diary, which was part of the crisis plan encouraging participants to monitor their suicidal thoughts on a daily basis. For many participants online diary was frustrating as it was not representative of the variety of feelings they experience throughout the day and was also not practical: ''I struggled with the online diary…. I was asked to upload a picture representing my typical mood of the day….But finding a specific picture representing a specific feeling was not really feasible for me…..The options for feelings were also too generic. I feel variety of emotions during the day not only sad, angry or happy…..I felt I was not able to express myself there" Participant C

More directive approach
Several participants spoke about including more instructions and coaching in the sessions. This was often mentioned as a way of feeling more in control whilst implementing the intervention in their daily life: "There were many exercises…..and I needed to find out which one works better for me…. I didn`t quite catch that in the beginning…It worried me… I felt I was not in control….I think there could be more personalised guidance so that, it`s easier to find the right exercises" Participant I

Indications of effectiveness
Overall, the analyses showed indication for change in suicidal ideation, hopelessness and depression scores, but not for worrying and quality of life scores over time (see table 2). These improvements occurred in the intervention group as well as in the control group and there were no post-test differences between the two groups ( Figure 2).
A Bayesian two-way Repeated Measures ANOVA comparing the pre (T0) and post-test (T3) BSS scores of intervention and wait-list control group participants revealed indication for a time effect on suicidal ideation (BF 10 : 50.4), hopelessness (BF 10 : 14.44) and depression (BF 10 : 127.09) scores, but not for group or interaction effects (see table 3).
The interaction plot for the suicidal ideation scores (see figure 2) indicates slightly more improvement in suicidal ideation among those in the intervention group compared to those in the wait-list control group. However, this was not supported by the results of the Bayesian analyses.
We have repeated the Bayesian two-way Repeated Measures ANOVA on those who scored 20 and above on BSS scores only (N=7). The results indicated stronger indication for a time effect on suicidal ideation, and depression scores, but not for group or interaction effects (see table 5 in appendix C).
There were two exceptions. Among those with severe suicidal ideation, there was stronger indication for time and group effects on hopelessness scores (BF 10 :10.512). This means that it is strongly likely that among those with severe suicidal ideation in both groups, the hopelessness scores were

Discussion
This small randomized controlled trial was set out to investigate the feasibility, and to demonstrate indications of effect, of an e-mental health intervention for suicidal ideation which is culturally adapted according to the Turkish migrants in the Netherlands and in the UK. We included 18 participants and among those, 7 of them had substantial levels of suicidal thoughts at baseline.
Despite this, no one reported a risk for a suicide attempt and the safety protocol did not need to be activated. Although therapeutic benefits of the intervention were emphasised (e.g. feeling connected with the intervention and/or personal coach), the low scores on usability of the intervention showed a number of barriers compromising its usability in daily life. Those with mild suicidal ideation and those who were not relating to the culturally adapted content (e.g. specific cultural case examples) emphasised not being represented by the content of the intervention. Further, there was no indication that the intervention group lead to better health outcomes than the control group. In both groups there was a reduction in suicidal ideation, depression and hopelessness, but not in worrying and quality of life.

Participants in the study
The reasons for the low up-take of the intervention remain unclear. One possible explanation is the difficulty in engaging with the target population during the recruitment process. In line with the trial of Ünlü İnce and colleagues (70), various channels have been used during the recruitment process such as social and mainstream media promotion, and TV programmes. Additionally, public events have been organised such as exhibition and film discussion. The rationale for using the latter methods was the growing evidence supporting the value of them in engaging with the target group concerning topics related with stigma and shame (71). Even though such events were organized in collaboration with community organisations and were usually well-attended, they did increase the publicity but did not often result in more participants.
Lack of anonymity during the recruitment process might be another important barrier. As part of the safety protocol, we collected personal information (name, address) and GP details during the recruitment process. Qualitative evidence on cultural meaning of suicide among Turkish migrants indicate that disclosing suicidal thoughts might mean dishonouring family and community by failing their expectations (9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21). Thus, the fear of disclosing identity might not have been eliminated during the recruitment process. Paper consent procedures in this current study might have contributed to this fear.
Further, it might also be that our target population was reserved in terms of seeking help for suicidal thoughts. Although the risk for suicidal behaviours is well-documented among Turkish populations (10), their mental health literacy (i.e. recognizing suicidal thoughts and symptoms of psychological distress) might be low. Support for the deleterious impact of the low mental health literacy on participant engagement comes from a recent pilot study investigating the feasibility of a mobile app in treating depression among Hispanic population in the States (35). Low mental health literacy was identified as a barrier restricting the up-take of the online app in this community (35). Although appealing, the mental health literacy was not investigated in our study. Thus, its impact in the implementation of our intervention remains unclear.
There might be other human factors contributing to the low up-take of the intervention. For instance, the target population might not have wanted an online intervention. The negative general perception of online interventions (e.g. they might be more appropriate to people from higher social class or those who are younger or more educated) was an important barrier hindering the engagement with the target population in a pilot RCT in Lebanon (39). Another attitudinal barrier might be about participating in an experimental study (with lots of extra questionnaires and the chance of entering the control group). Participating in research might precipitate the fear that their personal information might be used against them by the institutions of the host countries such as not hiring them for a job position because of having suicidal thinking or symptoms of depression (72).

Feasibility
Feasibility issues point to the barriers during the implementation of the intervention in real life. One of the main barriers was not feeling represented, despite the cultural adaptations of the intervention content. Tseng has argued for ensuring philosophical adaptations are also considered, that is the meaning of therapy and a therapeutic relationship, alongside technical, theoretical and practical modifications (73). More specifically, the characters representing Turkish migrants in the case examples were defined as more traditional in adherence to cultural and religious practices than the participants, and less educated than some participants. It could be that some of the participants in our sample had different cultural values and norms, emphasising caution in cultural adaptations to suit a range of demographic influences and cultural transition or acculturation.
The main reason for contextualizing the original intervention according to the Turkish cultural norms and values was to reach out to those who are more attached to these values (i.e. more traditional) in particular. Research from the Netherlands (66), Belgium (74) and Germany (75) emphasise that Turkish migrants who are more traditional, are more at risk for suicidal behaviours and are less likely to access the available treatment. It could be that the online form of delivery of the intervention was not suitable for this group specifically, or that the groups in the Netherlands are more traditional than the majority of participants who were from the UK. This necessitates a measure of acculturation in future research studies and adaptation to suit a variety of cultural groups.
There is a preliminary evidence for the acceptability of a culturally adapted group intervention targeting first generation Turkish migrants in the UK (76). Similar to the content of our online intervention, Perry and colleagues adapted the context of their intervention by incorporating idioms of distress and cultural case examples (76). Their intervention was delivered in a group format in a community setting which was accessible especially for the first generation migrants (76). Since the majority of participants considered the adaptations appropriate to the therapeutic needs of the traditional group of participants, it could be that not the content of adaptations but the format of delivery restricted the engagement with this group of participants.
It could also be that in our study, the cultural adaptations on the context of the intervention underscored the diversity of the Turkish migrant populations in Europe. This finding points to the "overgeneralisation" as one of the main challenges encountered in the cultural adaptation literature (24). Theoretically, it could be argued that even though migrants might share the same cultural identity (e.g. ethnicity, nationality and social group), there might be differences in micro-identities (e.g. political views, religion, sexual orientation) embedded within their cultural identities (77). The settlement and acculturation processes might complicate the formation of these identities even more so when the transnational migrant populations, such as Turkish migrants, are concerned. For instance, there has been an increase in political refugees from Turkey in West Europe (e.g. Germany, the Netherlands, and the UK) in recent years (78). This new group of Turkish migrants might have different micro-identities compared to the Turkish migrant populations with a long-standing history of settlement who have strong ties with their heritage culture (79). Even though we do not have information about the migration history of the participants, it might be that our sample was more representative of the recent group and this might have restricted the usability of the intervention in their daily life.
Another reason for feeling not represented was identified as not having severe suicidal thoughts. It could be that our intervention was more suitable to those with more severe suicidal ideation. Support for this comes from the results of our sensitivity analyses. We found stronger indications for an improvement in suicidal ideation, depression and hopelessness scores in both groups when we restricted the analyses with those with severe suicidal ideation scores. We also found stronger indication for a greater improvement in quality of life among those with severe suicidal ideation in the intervention group compared to those with severe suicidal ideation in the waitlist control group. These findings suggest that our inclusion criteria (scoring 1 and above on BSS) was too broad. This is discrepant from the trial of van Spijker and colleagues (29) where severe levels of suicidality and depression were associated with poor motivation (29). These differences could be explained with the cultural differences in help-seeking patterns. For instance, it is well-known that Turkish migrants tend to delay help-seeking until the symptoms of distress are more severe (65)(66)(67)(68)(69)(70)(71)(72)(73)(74)(75)(76)(77)(78)(79)(80). It could be that when the psychological distress caused by suicidal thoughts is unbearable then stigma and shame attached to suicide might have less influence on their motivation to seek help. However, this remains inconclusive for a number of reasons. First of all, we did not measure the mental illness stigma among participants. We also did not investigate whether there was a relationship between intensity of suicidal ideation, stigma and help-seeking for suicide among participants. Additionally, we did not have a sufficient number of participants to consider this trial definitive and assess effectiveness.
Feasibility issues might also be related with the delivery of the intervention. Some participants recommended a more directive approach giving them more clear instructions in following the sessions. Our intervention was set out with a lot of different exercises and participants were encouraged to choose the appropriate ones for them. It might be argued that more choice is not always the best option especially for some ethnic groups who are seeking explicit advice and assertion in their help-seeking process (81).

Effects
One of the obvious reasons for not finding an indication for the effect of the intervention was the small sample size. However, since the scores of the two groups were almost identical we do not expect that a larger trial will reveal effects. Both groups improved over time.
The finding that there was an improvement in suicidal ideation scores and in depression in the control group is in line with the results of the other trials (28-30). One of the explanations could be the effect of the safety protocol itself (i.e. being measured bi-weekly, having a safety procedure installed when necessary) in the waitlist control group during the first 6 weeks period upon randomization. It could be that safety protocol functions as an intervention in itself in the waitlist control group, and might be separately investigated. There is growing evidence indicating the value of safety planning as a standalone psycho-educational intervention increasing awareness about crisis situations, warning signs and available services for further help (82).
Alternatively, improvements in both groups can be attributed to receiving usual care which cannot be ruled out in the present study. Even though the majority of participants in both groups indicated not receiving any other help at baseline, their exposure to other sources of information and/or help was not carefully monitored throughout the study. This is an important methodological limitation (29-30-35). Future studies could benefit from an attention-control condition to monitor the elements and effects of usual care (30). Another possibility might just be the passing of time in that many states of distress are self-limiting and people might just spontaneously recover (83).
Notwithstanding with these limitations, this study has various strengths. An important strength is the safety of our intervention. The safety protocol did not need to be activated because of a risk for a suicide attempt during the study. None of the participants reported any adverse effects of the intervention, such as increased intensity of suicidal thinking, as a result of their participation. The safety of the e-mental health intervention is in line with the other trials investigating its effectiveness in general populations in the Netherlands (28), in Belgium (30) and in Australia (29). Thus, vulnerable groups such as minority groups with severe mental health symptoms could be studied safely, but a sub-group needs to be better identified for maximal benefit. Importantly, others may prefer alternatives, so e-interventions cannot address the needs of all.
Another strength is that the study followed a particular theoretical framework and a systematic approach to adapt the intervention according to the cultural beliefs and attributions about suicide and help-seeking for suicide among this high-risk group. The challenges encountered during the adaptation process such as overgeneralisation, highlight the importance of investigating the components of effective cultural adaptation further. For instance, when the context of the intervention is adapted for a particular sub-group within the target population, the fidelity of the intervention might be compromised and thus it might not be effective.
It is also noteworthy that the majority of the participants emphasised the therapeutic effect of guidance. Guidance was often considered as helpful in terms of feeling connected with the intervention. One of the notable recommendations of participants was about incorporating more guidance in the intervention. The expectation for more directive approach is in line with the cultural adaptation practices for Muslim populations (81). The refusal to give explicit advice or lacking assertion has been associated with incompetence and indecisiveness of the mental health professional, which might lead to the patient becoming irritated or discontinuing their therapy (81).
This finding supports the rationale of the decision for incorporating guidance into the e-mental health intervention in the present study as being different to the other trials (28-30-84).

Implications And Conclusions:
To conclude, the present study is an important step to further the current knowledge on whether online interventions could provide a feasible and an effective alternative in more complex contexts, including ethnic minority groups who are at elevated risk for suicidal behaviours. Our results suggest that there are number of considerations to incorporate into the intervention itself and into the design before further effectiveness testing. One of the key lessons for future studies is to investigate which components of cultural adaptation are effective in improving engagement with the target group. Our results suggest that adaptations in the context of the intervention should not exclusively represent the norms and values of the settled community but also the values of those who are in the process of settlement. Another key lesson for optimising the intervention further for the Turkish migrant populations is to structure the content of the intervention more. This might be important especially for ethnic groups who seek for instructions in their help-seeking process. Since the intervention was safe to use in our study, as well as in other trials, the intervention can be implemented through anonymous online platforms such as Samaritans in the UK and 113 online in the Netherlands. This might remove barriers for instance, fear of disclosing identity and fear of dishonouring one`s family, which were highlighted as potential barriers hindering participation in the present study. Since the participants valued the process of guidance, the online intervention might be well suited as an addon intervention to regular psychotherapeutic treatment. The effectiveness of such an additional intervention to regular face to face treatment should be studied carefully.
To conclude, the challenges encountered in this pilot study can be viewed as part of the incremental steps necessary to build future success in implementing online interventions among ethnic minorities in treatment of suicidal behaviours. This study and the previous trials seem to suggest the intervention needs further work and refinement and there should be more feasibility and exploratory trials of a modified intervention, refine it iteratively with feedback.

Consent for publication
Not applicable.

Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to them containing information that could compromise research participant privacy/consent but are available from the corresponding author on reasonable request with permission of the Department of Clinical Psychology, VU Amsterdam University.  Table 3. Themes related to views on the internet intervention (Kıyma Canına) and its adapted content N=12 (9 women, 3 men) of Turkish descent, and aged 23-56, in the Netherlands and in the UK   The interaction plot between group status and time (pre-test, week 2, week 4 and post-test) indicating changes in suicidal thinking in intervention group when compared with the waitlist control group

Supplementary Files
This is a list of supplementary files associated with this preprint. Click to download.