Study Protocol | Open | Open Peer Review | Published:
On the Road to Recovery psychological therapy versus treatment as usual for forensic mental health patients: study protocol for a randomized controlled feasibility trial
Pilot and Feasibility Studiesvolume 4, Article number: 124 (2018)
On the Road to Recovery (OTRTR) is a brief low intensity group psychological therapy that aims to improve patients’ insight into their mental disorder and develop adaptive coping skills to help manage distress. OTRTR is currently delivered in forensic mental health services in Scotland. However, to date, this therapy has not been evaluated as to its effectiveness or safety for forensic patients.
This is a parallel-group feasibility randomized controlled trial with single-blind assessments comparing OTRTR therapy to treatment as usual (TAU) for forensic mental health patients. Fifty participants will be recruited from high, medium, and low secure forensic mental health services in Scotland. Participants will receive OTRTR for approximately 12 weeks or continue treatment as usual for 12 weeks. The trial’s primary aims relate to testing the acceptability and feasibility of key trial procedures that would be necessary for a definitive RCT of OTRTR. The secondary aims include estimating therapeutic effect sizes on clinical outcomes including insight and coping skills. The study design also features an adverse event monitoring plan to estimate the safety of OTRTR for participants, as well as use of intensive longitudinal methods to identify “key ingredients” to the OTRTR therapy protocol.
This study will inform the design and sample size for a future full-scale randomized controlled trial (RCT), which will be conducted to determine the effectiveness of the On the Road to Recovery intervention in improving forensic mental health patients’ clinical insight and coping skills.
ISRCTN Registry, ISRCTN75126867 registered 27 July 2017
Forensic mental health patients often present with varied and complex treatment needs relating to their mental health and offending behavior [1, 2]. Treating these individuals requires significant involvement of clinical and forensic psychological practitioners as part of the multi-disciplinary treatment team. However, the evidence base for psychological interventions with forensic patients remains severely limited. Previous reviews have highlighted that a lack of rigorous methodologies, including the randomized controlled design, in forensic mental health evaluations has restricted the synthesis of effectiveness evidence in this field [2, 3].
Within NHS Scotland, psychological treatment of forensic patients is guided by the Forensic Matrix (an addendum to the larger “Matrix,” a guide to delivering evidence-based psychological therapies in Scotland ). The Forensic Matrix proposed a model of “stepped care”  for forensic patients. This model acknowledges that forensic patients often present with complex and enduring problems relating to their risk and offending behaviors which require highly specialist and individually tailored psychological treatment from practitioners with high levels of training and forensic expertise. However, they often also have simpler underlying or associated psychological needs which may respond to less intensive interventions. According to the Forensic Matrix stepped care approach, different levels of intervention intensity can be used to address forensic patients’ needs. For example, a forensic patient may require psychological work aimed at improving their management of basic emotions (low intensity) before the patient will be able to cope with the demands of sex-offending treatment (specialist/highly specialist). Several psychological interventions which address different common underlying needs of forensic patients have now been developed as part of the Forensic Matrix suite of interventions. There is now a critical need to evaluate the effectiveness of these therapies.
On the Road to Recovery (OTRTR) is a brief, low intensity group psychological intervention delivered across the Scottish Forensic Mental Health Managed Care Network (“Forensic Network”). The OTRTR protocol was written in 2011 by a working group of clinical and forensic psychologists within the Forensic Network. The program is based on the principles of cognitive behavior therapy with a focus on compassionate mind training . The core purpose of OTRTR is to open a dialog about recovery and instill in patients hope that recovery is possible. It is intended to be one of the first psychological treatments offered to patients on their journey to recovery from mental health difficulties, prior to engaging in more demanding and longer term therapy.
The aim of this study is to determine the feasibility of conducting an RCT comparing On the Road to Recovery psychological therapy to treatment as usual (TAU) for forensic mental health patients. Consistent with the UK Medical Research Council guidance (MRC) results from this study will directly inform the decision to undertake a future definitive RCT of OTRTR [4, 5]. Specifically, the feasibility RCT will determine:
Participant recruitment, consent, and retention rates
Acceptability of randomization to treatment condition
Acceptability of the OTRTR intervention for patients
Safety of the OTRTR intervention for patients
Acceptability of intensive longitudinal modeling procedures, in terms of:
The completion rate of standardized recording forms of the therapy content delivered by OTRTR therapy facilitators
The completion rate of primary clinical outcome measures weekly during treatment phase
Obtain estimates of likely therapeutic effects of OTRTR for use in sample size calculation of a future definitive trial. The between-group mean difference in therapeutic outcomes at the post treatment assessment, along with the standard deviation of the outcome for the entire sample, at baseline will be recorded.
This study is a feasibility study using a parallel-group randomized outcome blinded design to compare OTRTR to treatment as usual (TAU). As a feasibility study, the primary study aims relate to testing the acceptability and feasibility of key trial procedures that would be necessary for a full-scale RCT of this program. The comparator group will receive treatment as usual. In this study, 50% of participants will be randomized to commence OTRTR immediately for approximately 12 weeks. The remaining 50% will engage in TAU for 12 weeks and may complete OTRTR in the future. Participants will continue their usual access to concomitant therapies and therapeutic activities throughout the study. Participants will complete a range of self-report measures and a clinical interview at baseline (T0), post treatment (T1), and at follow-up (T2), 3-months after the T1 assessment, with an assessor who will be blind to participants’ allocation status. Clinical effect sizes will be estimated using two approaches, at the group level for a range of clinical outcomes by comparing change from pre to post treatment and at the individual level for the two primary outcomes—insight into mental disorder and use of adaptive coping skills—using intensive longitudinal methods (ILM) [7, 8]. Findings from two large treatment evaluations, the PEPS trial  and Ministry of Justice’s evaluation of the prison-based Core Sex Offender Treatment Programme , demonstrated that psychological therapies have the potential to cause harm to patients, a concept which has received little attention until recently. Accordingly, this study will pilot an adverse event monitoring plan to estimate the safety of OTRTR for participants.
Following participation in the study, participants will be offered the opportunity to complete a semi-structured interview which will gather qualitative information on their experiences of participating and views on different aspects of the trial (for instance, randomization, treatment, and assessments).
Participants will be recruited and identified from forensic mental health services which are part of the NHS Scotland. Services include high, medium, and low secure inpatient settings as well as community forensic teams.
As this study is a feasibility trial, a power calculation to determine sample size was not undertaken . The target recruitment for this feasibility trial will be N = 50 (25 in each intervention arm). This figure is consistent with recommendations for pilot and feasibility studies and appropriate given the study aims [12, 13].
People meeting inclusion criteria for the study include (1) males and females aged between 18 to 65 years, (2) who are receiving treatment under the Mental Health (Care and Treatment) (Scotland) Act 2003 from a participating forensic mental health service, and (3) who are viewed by their Responsible Medical Officer (RMO) as capable of providing informed consent and well enough to participate in the study.
People meeting exclusion criteria for the study include (1) those with a diagnosis of intellectual disability, (2) those who are viewed by their RMO as incapable of providing consent or are too unwell to participate, and (3) those who have completed On the Road to Recovery in the prior 3 years.
Identification and recruitment of participants
The study inclusion criteria were selected to match the broad range who are referred to On the Road to Recovery. Potential participants will be identified by their local psychological therapies team using the existing local referral process. The Responsible Medical Officers for each patient referred to On the Road to Recovery will be consulted on whether their patient has capacity to provide informed consent. Those without capacity will not be approached about the research. Potential participants with capacity to consent will be first informed of the study by their local psychological therapies team. Subsequently, a researcher will meet with these potential participants that have expressed an interest to provide them with further information and obtained informed consent to participate.
Sequence generation and allocation
Participants will be randomly assigned to either OTRTR or TAU with a 1:1 allocation ratio and varying block size (4 or 6), using a computer-generated randomization schedule stratified by site and assessed treatment need at T0. Participants will be categorized as having low or high treatment need based on scores on the two primary clinical outcome measures: the Birchwood Insight Scale and the Coping Styles Questionnaire. To categorize participants, we will employ cut-off scores of ≤ 9 (the cut-off for good versus poor insight following Birchwood et al. ) on the Birchwood Insight Scale and ≤ 37 on the Coping Styles Questionnaire  adaptive coping index (corresponding to 1 SD below the observed sample mean for State Hospital patients commencing On the Road to Recovery in a previous evaluation (McIntosh, Purcell, O’Rourke, Slesser, Thomson, manuscript in preparation)). Participants with poor insight and low adaptive coping skills will be considered to have high treatment need, and all other participants will be considered to have low treatment need. This stratum will be used to equally distribute patients with more acute needs across the two treatment conditions.
Staff of the Forensic Network office will generate the allocation sequence using an online randomization service (www.sealedenvelope.com). The Forensic Network office provides a centralized staff group who are knowledgeable of the participating services (the potential participating sites are part of the “Forensic Network”) but also independent from the trial research team. The Forensic Network office will generate the allocation sequences using an online randomization service (www.sealedenvelope.com). The researcher completing the baseline assessment will communicate the information necessary for allocation (site and assessed treatment need) to the Forensic Network office within 3 days of the participants’ T0 assessment. The Forensic Network office will allocate the participant and relay this information to the local investigators by email and keep a randomization record to audit compliance with the randomization procedure.
The assessor administering measures at T0, T1, and T2 assessments will be blind to participants’ allocation status. Participants and OTRTR facilitators will not be blind to allocation status. Participants will be asked not to divulge information about their group to the assessor. Any instances of unblinding will be recorded, and the clinical outcome analysis will be repeated excluding these participants to determine the robustness of the findings.
Description of the intervention
On the Road to Recovery
The OTRTR program is comprised of two brief treatment modules: Awareness & Recovery (A&R) and Looking After Yourself (LAY). A&R provides psychoeducational material adapted from Morrison et al.  “Think you’re crazy? Think Again” and from “Coping with Mental Illness” , the original psychoeducation program delivered across the Forensic Network. The LAY program’s core aim is to help patients develop basic coping skills to manage psychological distress and to develop a “toolkit” for coping with this distress now and in the future. The protocol is semi-structured and organized by topics instead of sessions, which allows the program to flexibly match pace to groups with greater or lesser need by adjusting the number of sessions. Each topic contains several objectives that are accompanied by structured didactic activities, exercises, and discussion prompts. The program will run over approximately 12 weeks, with weekly 2-h sessions. Treatment delivery is primarily in small groups; however, the program can also be delivered on an individual basis as part of the study. Participant attendance and engagement in the treatment sessions will be monitored and recorded. Participants’ RMOs may withdraw a participant from the study and therefore discontinue OTRTR sessions should there be a deterioration in participants’ clinical presentation.
OTRTR therapy facilitators will receive group supervision at least once every four sessions delivered, consistent with the Forensic Matrix governance standards . Facilitators will be provided with standardized recording documents to closely record the program content delivered in each session.
Treatment as usual
Participants randomized to the TAU condition will engage in treatment as usual including access to other psychological therapies, with the exception that they do not attend OTRTR sessions. TAU participants will be advised they will be offered the OTRTR therapy at a later stage. Access to and engagement in non-psychological therapies differs by participating site and individual participants. For both groups, we will record the care received by participants during the trial to enable a full description of treatment as usual.
We will monitor potential harm from study treatments experienced by participants by recording the occurrence of adverse (AE) and serious adverse events (SAE). Any AE and SAE experienced by participants after consenting to the study will be recorded and reported regardless of whether it is likely that they are the result of treatment received. The Health Research Authority defines SAE as an occurrence that (a) results in death, (b) is life-threatening, (c) requires hospitalization or prolongation of existing hospitalization, (d) results in persistent or significant disability or incapacity, and (e) consists of a congenital anomaly or birth defect. Applying this definition to psychiatric inpatient settings, acts of serious self-harm, attempted suicide, and suicide would qualify as a SAE. Events that constitute SAEs are already routinely collected and monitored at all participating sites using the Datix incident reporting software.
AEs will be similarly monitored throughout the study. In the present study, an AE is considered to have occurred if either of the following takes place: (a) participant is removed from the study at the request of their RMO due to significant deterioration in the patient’s mental state and/or behavior and (b) a participant’s global CORE-OM score (indexes overall psychiatric distress) increases from the previous assessment to an extent that is both clinically significant and reliable (as defined in Evans et al. ). All SAEs and AEs will be recorded, and SAEs will be reported to a NHS Scotland Research Ethics Committee if and when they occur.
Data collection and outcome measures
The trial consists of a 12-week treatment phase with a 3-month (13-week) follow-up phase. As shown in Fig. 1, participants in both treatment conditions will be assessed pretreatment at T0, post treatment at T1, and 3-months after post-treatment at T2. A schedule of study procedures and assessments is detailed in Table 1.
The following measures will be collected at T0. They will be used as moderator variables in analysis to identify mechanisms of change and predict therapy outcomes at the individual level. We will explore the associations between these variables and change on the clinical outcome measures from T0 to T1.
Demographic data (age, sex, ethnicity, education level)
Primary and secondary psychiatric diagnoses
Medication at each assessment point
Previous participation in psychological therapies
Time since admission (months)
Verbal working memory: The Letter-Number Sequencing subtest of the WAIS-IV  will be used to index verbal working memory. Working memory was identified as one of several cognitive abilities which may impact patients’ ability to engage with and benefit from the On the Road to Recovery intervention.
Primary outcome measures
Primary study outcomes relate to the feasibility and acceptability of key trial procedures.
Number of eligible participants identified over the study period will be indexed by the total number of participants identified across sites who meet eligibility criteria.
Estimate rate of recruitment into the trial
The proportion of eligible patients who agree to participate (provide consent): We anticipate the consent refusal rate to fall in the range of 16–37% based on findings from previous RCTs of psychological interventions in forensic populations [21,22,23,24]. The adopted benchmark of success in this study is a consent refusal rate in this study of less than 37%.
The number of participants enrolled into the study each month during the recruitment period: As in other ongoing feasibility RCTs , the benchmark of recruitment success is achieving 80% of target sample size, in this case recruiting at least 40 participants into the study.
Audit adherence to randomization procedure: The number of instances where the actual treatment allocation differed from assigned allocation. Instances where the participant is removed from the study by his or her RMO, who cites a reason related to the allocated condition as opposed to deterioration in clinical presentation, will also be considered non-adherence to the randomization process. Any instances of non-adherence to the randomization procedure will cause great concern as to the feasibility of a future phase III RCT of OTRTR.
Estimate completion rate of OTRTR: Proportion of participants who complete OTRTR. Following similar studies [26, 27] participants will be considered to have completed OTRTR if they attended at least 80% of the planned sessions.
Explore reasons for study drop-out during the study treatment: Participants will have the option of providing a reason they do not wish to continue with the study before exiting the study.
Completion rate of primary clinical outcome measures weekly during treatment phase: The proportion of Birchwood Insight Scale (BIS) and Coping Styles Questionnaire (CSQ) forms completed as intended each week during the treatment phase will be calculated.
Completion rate of standardized recording forms of the therapy content delivered by OTRTR therapy facilitators: Proportion of complete forms will be calculated.
Number of participants lost to follow-up and reasons: Number of participants who withdrew from the study or who were otherwise lost to follow-up, with reasons recorded (if provided). The adopted benchmark of success is a participant attrition rate of less than 20% at the post-treatment (T1) assessment.
Safety of OTRTR for patients: Descriptive statistics for frequency of observed AEs and SAEs across treatment conditions. Adverse events may occur for reasons independent of the therapy being studied due to the nature of this clinical population. Safety of OTRTR will be assessed by comparing the rates of SAEs and AEs across the two study treatment conditions. We will also compare the incident rate of SAEs for study participants to the rate observed for each participant in the 12 weeks prior to enrolment in the study. This reference data will be requested from the local site clinical effectiveness departments or equivalents following study completion.
Average duration of assessment battery at T0, T1, and T2. Descriptive statistics for length of assessment. We anticipate the T0 assessment will last approximately 90 min, and the T1 and T2 assessments 60 min each.
After T2 assessment, each participant will be offered the opportunity to complete a semi-structured interview with the researcher. Interviews will solicit participants’ experiences of participating in the study, with a focus on the treatment phase and assessments. These results will inform the design of a future RCT of OTRTR.
The secondary outcome measures relate to intended treatment outcomes of OTRTR. The following measures will be completed at baseline (T0), post treatment (T1), and 3-month follow-up (T2) by a researcher blind to the participant’s allocation status.
Birchwood Insight Scale (BIS)  is an eight-item self-report measure of insight. The original study established BIS total score test-retest reliability (1-week) as r = 0.90 and internal consistency of α = 0.75. The BIS variable used in this study will be the total score (max 12).
Coping Styles Questionnaire (CSQ)  is a 60-item self-report measure on the use of maladaptive and adaptive coping styles. It is comprised of four subscales, each a unique dimension of coping: rational, detached, emotional, and avoidance. Test-retest reliability of the CSQ subscales range from r = 0.70 to 0.80, and the subscale internal consistencies of α = 0.69 to 0.90. The CSQ variable used in this study will be the total of the two adaptive coping subscales, Rational and Detached.
Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM)  is a 34-item self-report measure of psychological distress that covers four domains: wellbeing, symptoms/problems, functioning, and risk to self and others. A global measure of distress is also calculated across these domains. Good psychometric properties for this measure in non-clinical, clinical, and forensic populations have been established [19, 29]. The CORE-OM variable used in this study will be the 34-item mean score (max 4).
Rosenberg Self-Esteem Scale (RSE)  is a 10-item self-report measure of self-esteem. The RSE has an internal consistency of α = 0.77 and two-week test-retest reliability of r = 0.85. The total RSE score will be used (max 30).
Questionnaire on the Process of Recovery (QPR)  is a 22-item measure of service user-rated recovery comprised of an interpersonal and intrapersonal subscale. The initial validation study found acceptable two-week test-retest reliability (intrapersonal subscale r = 0.87, interpersonal subscale r = 0.77) and internal consistency (intrapersonal α = 0.94, interpersonal α = 0.77). The intrapersonal (max 68) and interpersonal (max 20) subscale scores will be used.
Brief Psychiatric Rating Scale (BPRS)  is a clinician/researcher-rated tool used to assess severity of psychopathology. The 18 items sum to derive a continuous total score of symptom severity. The BPRS has good psychometric properties is the most widely used outcome measure in treatment studies across populations with severe and persistent mental illness . The total BPRS score will be used (max 126).
Institution-recorded incidents of physical aggression and violence: The number of incidents of violence/aggression, for which the participants are considered to have responsibility in instigating or exacerbating, during the study treatment phase and 3-month follow-up period will be compared across treatment groups. This information will be as participants complete T2.
Institutional privileges: Changes in institutional privileges (e.g., increased grounds access, unsupervised phone calls, and patient outings) will be recorded for all participants during the study treatment phase and 3-month follow-up period. This information will be after participants complete T2.
Weekly clinical measures
Following an intensive longitudinal methods (ILM) approach [7, 8], the BIS and CSQ will be administered to all participants each week during the 12-week study treatment phase. Members of the local health care team will administer these weekly measures and as such they will not be blind to participants’ treatment allocation status.
Participant and data retention
Participants may withdraw from the study at any time. We will offer participants who choose to withdraw from the study the opportunity to share their views and reasons for withdrawal in a post-study interview at the time of their exit from the study. Participants will be reassured that this is voluntary, and they need have no further contact with the research team. Discontinuing OTRTR therapy will not exclude participants from the study. Participants who wish to discontinue OTRTR will be offered the opportunity to participate in the regular assessment points should they wish. Participants who withdraw from the research study will be able to continue (if in OTRTR condition) or commence (if in TAU condition) OTRTR in the context of their usual care and treatment. Measures will be clearly prioritized so that in all assessment batteries primary measures will be administered prior to secondary minimizing the loss of primary data should participants withdraw during an assessment session. The assessment battery may be completed over two sessions to avoid participant testing fatigue.
Any modifications to the protocol that may impact on the conduct of the study will require a formal amendment to the protocol. Such amendments will be agreed upon by the trial research team, and approved by NHS Scotland Research Ethics Committee prior to implementation, and notified to the local research committees of participating organizations in accordance with local regulations. Minor corrections and/or clarifications to the protocol, including administrative changes that have no effect on the way the study is to be conducted, will be agreed upon by the trial research team and documented. NHS Scotland Research Ethics may be notified at the discretion of the trial research team. The ISRCTN trial registration will also be updated to reflect any protocol amendments made.
All research-generated data will be entered into password-protected electronic spreadsheets and stored separately from any participant identifying information. Participants will be assigned unique study IDs. A master list matching participant names and study IDs will be kept in a password-protected excel document and stored on a secure NHS drive. Only members of the trial research team will have access to the master list. Original forms (e.g., questionnaires) will be retained for a period of 3 years after completion of the study, stored securely on NHS Scotland premises. Anonymized data files may be retained indefinitely by the research team. To ensure accurate data collection, the optional post-study interviews will be audio recorded and transcribed for qualitative analysis. Audio data will be collected using an encrypted password-protected audio recorder and uploaded to a secure NHS drive. Data entry, preparation, and analysis will be carried out using a bespoke standardized operating procedure to ensure data integrity.
Feasibility and acceptability outcomes
Descriptive statistics will be reported for the key outcomes on the feasibility data (including mean averages, standard deviations, and ranges where appropriate). The number of patients who were eligible, recruitment into the study, and attrition rates according to each intervention arm will be considered using descriptive statistics and reported using the CONSORT (Consolidated Standards of Reporting Trials) flow chart (Fig. 1). Interviews will be analyzed using thematic analysis  to identify emerging themes within the data.
This study will not be sufficiently powered to determine treatment efficacy. Evidence of change on clinical outcomes will be considered with a focus on effect sizes and reliable change indices over null hypothesis significance testing. Effect size estimates (Cohens’ d) will inform sample size estimates and inform decisions regarding the most appropriate outcome measures for use in a future phase III trial of OTRTR. A modified intention to treat (ITT) analysis will be used to analyze change on the outcome measures and derive the effect size estimates. Therapeutic effect sizes (with 95% confidence intervals) will be derived using linear regression models comparing T1 scores for the two treatment arms, with baseline (T0) scores included as covariates. These analyses will be repeated using between-group comparison of means at T2 for estimated maintenance of treatment effects.
Similarly, effect sizes for change in institutional violence/aggression will be derived. The mean number of incidents of violence/aggression (recorded by the institution) measured at post treatment (the mean number of incidents for duration of study treatment phase, up to T1) will be compared, controlling for baseline differences (the mean number of incidents for the 3-months preceding treatment, up to T0). This analysis will be repeated using between-group comparison of mean incidents during follow-up period (record from T1 to T2) for estimated maintenance of treatment effect.
Changes in institutional privileges (e.g., increased grounds access, unsupervised phone calls, and patient outings) will be recorded for participants in both groups throughout the study treatment phase (T0 to T1) and follow-up period (T1 to T2). To the researchers’ knowledge, this type of outcome has not been previously studied in treatment evaluations in forensic settings. This data will be examined primarily to identify the types of changes that occur and will be used to inform secondary outcome measure selection and refinement in a future definitive RCT. Descriptive statistics will be calculated and examined.
An exploratory analysis will also estimate the individual-level effects on the two primary clinical outcome measures (Birchwood Insight Scale and the Coping Styles Questionnaire). This analysis will use the weekly BIS and CSQ scores collected during the study treatment phase and analyze change using multilevel modeling (also referred to as linear mixed effects modeling). The multilevel approach is advantageous as it accounts for dependencies among observations (repeated assessments) nested within individuals and is flexible in the presence of missing data. The analysis will reveal individual trajectories (e.g., a timecourse of therapeutic effect) and variability throughout the treatment period. The model will include two levels: participants and time (repeated observations), where time is nested within each participant. Level of treatment need (assessed at baseline) will be an additional model parameter. As for the between-group analyses which aim to estimate treatment effects, this individual-level treatment effects analysis will not be sufficiently powered for definitive hypothesis testing and results will be interpreted with caution.
To our knowledge, there is no recommended Minimal Clinically Important Difference (MCID) for this study’s primary outcome measures in forensic psychiatric populations: the Birchwood Insight Scale and the Coping Styles Questionnaire. Results from this study will be used to better understand what an appropriate MCID would be for this measure in this population. We will adopt a distribution-based approach, which compares change on the outcome measure to an index of the measure’s variability [35, 36]. This approach assumes that the outcome measure change score equivalent to a small effect size (e.g., Cohen’s d = 0.2) would be the MCID. The MCID will be calculated multiplying 0.2 by the standard deviation of the outcome measure across the feasibility study’s entire sample at baseline.
When data is missing, we will record reasons why missing data occurred, which will help to formulate assumptions about the missing data (e.g., missing at random) for the purpose of analysis. We will use a model-based multiple imputation method to address any missing outcome, though the exact method will be selected after a survey of the pattern of missing data.
Using the standardized recording forms completed by the therapy facilitators, each therapy session will be retrospectively coded as focusing primarily on either (1) building insight or knowledge of mental health or (2) developing coping skills. In a separate model restricted to only OTRTR condition participants, therapy session focus (measured using the standardized recording forms completed by the OTRTR facilitators) will be further included as an additional predictor of change on the clinical outcome measures.
Data and safety monitoring issues will be reviewed on a quarterly basis by the trial research team comprised of LT, SOR, and LGM. Urgent issues will be addressed as soon as possible via phone, email, or regular project meetings that include the trial research team.
A separate formal data monitoring committee will not be convened for this study due to limited resources and the feasibility nature of this study.
Determining progression to a full trial
There are specified criteria of success in relation to participant consent (refusal < 37%), recruitment (recruit at least 40 participants) and retention (attrition < 20% at T1). Should these criteria not be met, we would not progress to a full trial using the same design. Further feasibility outcomes have been identified as critical for the decision to pursue a future RCT of OTRTR at all. For example, if this study finds significant problems with the application of the randomization procedure (e.g., non-adherence at the participating sites), this would indicate a randomized design comparing OTRTR to any comparison treatment is not feasible and should not be pursued in this treatment setting. However, most other potential feasibility or acceptability issues found by this study could be addressed by modifying the relevant procedures in a larger RCT. If significant modifications were indicated, we would consider the need for a second feasibility trial or an embedded pilot within a full-scale RCT. Effect size estimates from this feasibility study may inform the power calculation of a future RCT only if there have been minor changes to the study design between the two studies. In this case, when calculating the necessary sample size of a future trial, we would consider both the effect sizes observed in the feasibility study (using lower bound 95% confidence interval) and the minimum clinically important difference in the primary outcomes.
Trial results will be communicated in conference presentations, peer-reviewed journal publications and LGM’s PhD dissertation. Consistent with the University of Edinburgh Open Access policy, findings will either be published in Open Access journals or a copy of the publication will be deposited in an Open Access Repository. Results will be disseminated in manner that maintains the confidentiality of individual participants. A summary of the results will be provided to those participants who express an interest.
Forensic mental health patients are often complex individuals with significant treatment needs relating to their mental health and risk. Psychological care addressing these needs must be carefully coordinated and sequenced often over a period of several years. The OTRTR psychological intervention program was developed in Scotland as an initial treatment program for patients near the start of their recovery journey with forensic mental health services. A full evaluation of OTRTR’s effectiveness is necessary to inform evidence-based clinical decision making. UK Medical Research Council guidance [37, 38] emphasizes the need to thoroughly test the feasibility and acceptability of key trial procedures prior to commencing a definitive effectiveness trial.
This study is not without its limitations. The nature of the comparison condition can greatly affect RCT outcomes and deserves substantial consideration in the design phase . While a study-defined comparison condition was initially preferred by the researchers, this raised ethical and organizational concerns and we ultimately made a pragmatic decision to utilize a routine care comparison condition. Details of the care accessed by study participants will be recorded to enable a full description of what routine care entailed.
As in many evaluation studies of psychological therapies, there is a difficult balance of protocol fidelity and ensuring the treatment is responsive to participants’ needs. The multi-site nature of this evaluation increases the likelihood of variation in treatment content delivered throughout the study. However, we intend to statistically model the effects of this variation in a novel exploratory analysis using multilevel modeling. We anticipate that this method will be useful in allowing a degree of treatment flexibility within a rigorous design such as the RCT. Moreover, the ability to statistically model changes in key clinical outcomes to particular elements of the therapy protocol will add helpful insights and increase the generalizability of any OTRTR clinical effects observed.
Awareness and Recovery
Birchwood Insight Scale
Brief Psychiatric Rating Scale
Consolidated Standards of Reporting Trials
Clinical Outcomes in Routine Evaluation – Outcome Measure
Coping Styles Questionnaire
Intensive longitudinal methods
Intention to treat
Looking After Yourself
Medical Research Council
National Health Service
On the Road to Recovery
Questionnaire on the Process of Recovery
Randomized controlled trial
Research ethics committee
Responsible Medical Officer
Rosenberg Self-Esteem Scale
Serious adverse event
Treatment as usual
Barnao M, Ward T. Sailing unchartered seas without a compass: a review of interventions in forensic mental health. Aggress Viol Behav. 2015;22:77–86. https://doi.org/10.1016/j.avb.2015.04.009.
Sturgeon M, Tyler N, Gannon TA. A systematic review of group work interventions in UK high secure hospitals. Aggress Viol Behav. 2017;38:53–75. https://doi.org/10.1016/j.avb.2017.11.004.
Duncan EAS, Nicol MM, Ager A, Dalgleish L. A systematic review of structured group interventions with mentally disordered offenders. Crim Behav Ment Health. 2006;16:217–41. https://doi.org/10.1002/cbm.631.
NHS Education for Scotland. The Matrix: a guide to delivering evidence-based psychological therapies in Scotland. 2015. http://www.nes.scot.nhs.uk/education-and-training/by-discipline/psychology/the-matrix-(2015)-a-guide-to-delivering-evidence-based-psychological-therapies-in-scotland.aspx. Accessed 16 Sept 2016.
Bower P, Gilbody S. Stepped care in psychological therapies: access, effectiveness and efficiency. Brit J Psychiat. 2005;186:11–7. https://doi.org/10.1192/bjp.186.1.11.
Gilbert P, Irons C. Focused therapies and compassionate mind training for shame and self-attacking. In: Gilbert P, editor. Compassion: conceptualisations, research and use in psychotherapy. London: Routledge; 2005. p. 263–325.
Bamberger KT. The application of intensive longitudinal methods to investigate change: stimulating the field of applied family research. Clin Child Fam Psych. 2016;19:21–38. https://doi.org/10.1007/s10567-015-0194-6.
Shadish WR, Kyse EN, Rindskopf DM. Analyzing data from single-case designs using multilevel models: new applications and some agenda items for future research. Psychol Methods. 2013;18:385–405. https://doi.org/10.1037/a0032964.
McMurran M, Day F, Reilly J, Delport J, McCrone P, Whitham D, et al. Psychoeducation and Problem Solving (PEPS) therapy for adults with personality disorder: a pragmatic randomized-controlled trial. J Personal Disord. 2017;31:810–26. https://doi.org/10.1521/pedi_2017_31_286.
Mews A, Di Bella L, Purver M. Impact evaluation of the prison-based core sex offender treatment programme. Ministry of Justice. 2017. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/623876/sotp-report-web-.pdf. Accessed 6 Sep 2017.
Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1. https://doi.org/10.1186/1471-2288-10-1.
Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4:287–91. https://doi.org/10.1002/pst.185.
Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10:307–12. https://doi.org/10.1111/j..2002.384.doc.x.
Birchwood M, Smith J, Drury V, Healy J, Macmillan F, Slade M. A self-report insight scale for psychosis: reliability, validity and sensitivity to change. Acta Psychiat Scand. 1994;89:62–7. https://doi.org/10.1111/j.1600-0447.1994.tb01487.x.
Roger D, Jarvis G, Najarian B. Detachment and coping: the construction and validation of a new scale for measuring coping strategies. Personal Individ Differ. 1993;15:619–26. https://doi.org/10.1016/0191-8869(93)90003-L.
Morrison AP, Renton J, French P, Bentall R. Think you're crazy? Think again: a resource book for cognitive therapy for psychosis. 1st ed. London: Routledge; 2008.
Walker H, Tulloch L, Ramm M, Drysdale E, Steel A, Martin C, et al. A randomised controlled trial to explore insight into psychosis; effects of a psycho-education programme on insight in a forensic population. J Forens Psychiatry Psychol. 2013;24:756–71. https://doi.org/10.1080/14789949.2013.853821.
Forensic Mental Health Services Managed Care Network. The Forensic Mental Health Matrix. 2011. http://www.forensicnetwork.scot.nhs.uk/. Accessed 16 September 2016.
Evans C, Connell J, Barkham M, Margison F, McGrath G, Mellor-Clark J, Audin K. Towards a standardized brief outcome measure: psychometric properties and utility of the CORE-OM. Brit J Psychiat. 2002;180:51–60. https://doi.org/10.1192/bjp.180.1.51.
Wechsler D, Coalson DL, Raiford SE. WAIS-IV: Wechsler adult intelligence scale. San Antonio: Pearson; 2008.
Aho-Mustonen K, Tiihonen J, Repo-Tiihonen E, Ryynänen OP, Miettinen R, Räty H. Group psychoeducation for long-term offender patients with schizophrenia: an exploratory randomised controlled trial. Crim Behav Ment Health. 2011;21:163–76. https://doi.org/10.1002/cbm.788.
Cavezza C, Aurora M, Ogloff JR. The effects of an adherence therapy approach in a secure forensic hospital: a randomised controlled trial. J Foren Psychi Psych. 2013;24:458–78. https://doi.org/10.1080/14789949.2013.806568.
Doyle M, Tarrier N, Shaw J, Dunn G, Dolan M. Exploratory trial of schema-focussed therapy in a forensic personality disordered population. J Foren Psychi Psych. 2016;27:232–47. https://doi.org/10.1080/14789949.2015.1107119.
Kuokkanen R, Lappalainen R, Repo-Tiihonen E, Tiihonen J. Metacognitive group training for forensic and dangerous non-forensic patients with schizophrenia: a randomised controlled feasibility trial. Crim Behav Ment Health. 2014;24:345–57. https://doi.org/10.1002/cbm.1905.
Jacobsen P, Peters E, Chadwick P. Mindfulness-based crisis interventions for patients with psychotic symptoms on acute psychiatric wards (amBITION study): protocol for a feasibility randomised controlled trial. Pilot Feasibility Stud. 2016;2:43. https://doi.org/10.1186/s40814-016-0082-y.
Cullen AE, Clarke AY, Kuipers E, Hodgins S, Dean K, Fahy T. A multisite randomized trial of a cognitive skills program for male mentally disordered offenders: violence and antisocial behavior outcomes. J Consult Clin Psych. 2012;86:1114–20. https://doi.org/10.1037/a0030291.
Yip VC, Gudjonsson GH, Perkins D, Doidge A, Hopkin G, Young S. A non-randomised controlled trial of the R&R2MHP cognitive skills program in high risk male offenders with severe mental illness. BMC Psychiatry. 2013;13:267. https://doi.org/10.1186/1471-244X-13-267.
Evans C, Mellor-Clark J, Margison F, Barkham M, Audin K, Connell J, McGrath G. CORE: clinical outcomes in routine evaluation. J Ment Health. 2000;9:247–55. https://doi.org/10.1192/bjp.180.1.51.
McCloskey H. Evaluation of the CORE outcome measure in a therapeutic forensic setting. Brit J of Forensic Pract. 2001;3:22–8. https://doi.org/10.1108/14636646200100011.
Rosenberg M. Society and the adolescent self-image. Princeton: Princeton; 1965.
Neil ST, Kilbride M, Pitt L, Nothard S, Welford M, Sellwood W, et al. The questionnaire about the process of recovery (QPR): a measurement tool developed in collaboration with service users. Psychosis. 2009;1:145–55. https://doi.org/10.1080/17522430902913450.
Overall J, Gorham D. The brief psychiatric rating scale. Psychol Rep. 1962;10:799–812. https://doi.org/10.2466/pr0.1918.104.22.1689.
Burlingame G, Dunn T, Chen S, Lehman A, Axman R, Earnshaw D, Rees FM. Selection of outcome assessment instruments for patients with severe and persistent mental illness (SPMI). Psychiatr Serv. 2005;56:444–51. https://doi.org/10.1176/appi.ps.56.4.444.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.
Copay AG, Subach BR, Glassman SD, Polly DW, Schuler WC. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J. 2007;7:541–6.
Angst F, Aeschlimann A, Angst J. The minimal clinically important difference raised the significance of outcome effects above the statistical level, with methodological implications for future studies. J Clin Epidemiol. 2017;82:128–36.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. https://doi.org/10.1136/bmj.a1655.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. https://doi.org/10.1136/bmj.h1258.
Mohr DC, Spring B, Freedland KE, Beckner V, Arean P, Hollon S, et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother Psychosom. 2009;78:275–84. https://doi.org/10.1159/000228248.
Thank you to Professor Matthias Schwannauer and Dr. Mandy Johnstone for their critical review and comments on the study protocol. Thanks also to the Forensic Matrix Working Group members for their support of the study and feedback on its design.
The study is currently operating on study protocol version 1.0 dated 30 March 2017.
The Forensic Mental Health Services Managed Care Network, located within the State Hospital in Carstairs, provided funds for this work to be conducted as part of LM’s doctoral degree. The Forensic Network has not had a role in the design of the study or development of the protocol.
Availability of data and materials
Once data are collected, the datasets used and/or analyzed during the current study will available from the corresponding author on request.
Ethics approval and consent to participate
Ethical approval for this research study has been obtained from NHS South East Scotland Research Ethics Committee 01, REC reference: 17/SS/0064. Informed consent will be obtained from all participants.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.