Skip to main content

Acceptability and proof of concept of internet-delivered treatment for depression, anxiety, and stress in university students: protocol for an open feasibility trial

Abstract

Background

In recent years, university counseling and mental health services have reported an increase in the number of clients seeking services and in yearly visits. This trend has been observed at many universities, indicating that behavioral and mental health issues pose significant problems for many college students.

The aim of this study is to assess the acceptability and proof of concept of internet-delivered treatment for depression, anxiety, and stress for university students.

Methods/design

The study is an open feasibility trial of the SilverCloud programs for depression (Space from Depression), anxiety (Space from Anxiety), and stress (Space from Stress). All three are 8-module internet-delivered CBT (iCBT) intervention programs. Participants are assigned a supporter who provides weekly feedback on progress and exercises. Participants will complete the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder-7 (GAD-7), and stress subscale of the Depression, Anxiety, Stress Scale-21 (DASS-21) as the outcome measures for the depression, anxiety, and stress interventions, respectively. Other outcomes include measures of acceptability of, and satisfaction, with the intervention. Data will be collected at baseline, 8 weeks and 3-month follow-up.

Discussion

It is anticipated that the study will inform the researchers and service personnel of the programs’ potential to reduce depression, anxiety, and stress in a student population as well as the protocols to be employed in a future trial. In addition, it will provide insight into students’ engagement with the programs, their user experience, and their satisfaction with the online delivery format.

Peer Review reports

Background

Depression, anxiety, and stress are among the primary causes of disease rates worldwide [22] and are the most prevalent mental health problems in the USA [23]. Each is associated with significant economic, personal, intrapersonal, and societal losses including lower quality of life and increased mortality [8, 35, 37].

Depression and anxiety are also the most prevalent mental health problems among the student population [10, 34]. In the 2010 American College Health Survey, 48 % of college and university students reported feeling overwhelming anxiety at least once in the previous year. Thirty-one percent reported feeling so depressed that it was difficult to function at least once in the previous 12 months [2]. More recently, university counseling and mental health services have reported an increase in the number of clients seeking services and in yearly visits [5].

The college years can be a highly stressful time in students’ lives. This is particularly true of the period of transition to college as students are learning to cope with increased academic pressures [27, 48]. Students are at a developmental stage when newfound stressors can promote the onset of mental health difficulties [12, 46, 47]. In addition to academic stress, international students can experience significant sociocultural adjustment demands [19].

Young adults between 17 and 25 years of age are reported to be at greater risk of developing a serious mental illness than individuals in other age groups [51]. Although early diagnosis of mental disorders can be difficult, delayed diagnosis can often lead to treatment resistance and poorer longer-term outcomes [47, 53]. In addition, underachievement or failure at this point in life can cause long-term setbacks to individuals’ self-esteem and future progress [21, 47].

Treating depression, anxiety, and stress

Depression, anxiety, and stress disorders can each be treated effectively using medications; however, after completing a course of this type of treatment, the chances of relapse are high, and equally successful psychological therapies are available [36]. Of these, cognitive-behavior therapy (CBT) is the most widely researched and CBT is recognized as the leading choice of treatment for depression and anxiety in terms of post-treatment improvements, maintaining progress, and preventing relapse [18, 30]. CBT also is effective for stress management [44]. CBT is comprised of a variety of cognitive and behavioral approaches, each concerned with changing distressing thoughts and beliefs. Treatment is often comprised of self-monitoring and thought recording, behavioral activation, cognitive restructuring, and exposure [11].

Access to treatment

On a global scale, a significant number of individuals in need of psychological treatment receive no medical diagnosis nor do they seek treatment [1]. The worldwide treatment gap between those needing treatment for depression compared to those receiving treatment has been estimated at 56 %; the gap in treatment for anxiety disorders has been estimated at 46 % [25]. Several factors prevent people from accessing treatment, such as waiting lists, lack of motivation, negative attitudes about treatment, and costs [25, 29]. Among those willing to seek treatment, many encounter a lack of trained professionals or are placed on a waiting list [47]. Although students can often obtain mental health services on campus, many campuses have limited services or have wait lists for services because of the high demand [24]. A survey of US students with mental health problems revealed that common barriers to seeking services, even if they were available, were stigma, lack of time/accessibility, and wait lists [15]. Internet-delivered interventions are one way to address these issues.

Internet-delivered CBT (iCBT)

Internet-delivered CBT programs are specially designed for the treatment of specific disorders and can be either clinician-guided or self-administered. When such interventions are used in the UK, 6–8 sessions are the recommended length of treatment [3133]. Many studies provide support for the utility and effectiveness of internet-delivered treatments for depression, anxiety, and stress [4, 10, 38]. Those with the additional feature of human support tend to produce better results [38].

Several studies also have assessed the efficacy of online interventions for depression, stress, and anxiety for college students. Recent reviews of these studies have found evidence for the efficacy of these online interventions [9, 14]. However, most of these studies do not assess the potential for change of the interventions in the context of existing service delivery systems. Rather, many are RCTs assessing efficacy in convenience samples of students (e.g., Psychology majors). The current trial seeks to investigate the acceptability and potential for change of internet-delivered interventions among university students seeking services and the procedures needed to incorporate these interventions into mental health delivery in US universities. Additionally, it hopes to support and extend existing empirical evidence for SilverCloud programs [4042].

Objectives of the trial

The aim of this study is to test the acceptability of internet-delivered treatment for depression, anxiety, and stress in university students. The trial will investigate the feasibility of recruiting students (in relation to numbers screened, numbers recruited, and level of adherence) to receive internet-delivered interventions for the treatment of depression, anxiety, and stress and establish the acceptability of the interventions to the students at University of Minnesota.

Acceptability of the intervention to clients will be assessed using data on usage and engagement with the intervention (e.g., percentage of participants completing each module, average number of logins, average time spent per session, and total time spent on the program). These data are acquired through the online SilverCloud system. Satisfaction will be assessed through the use of a post-intervention questionnaire on satisfaction with accessing and using an online delivery format for treatment. At 3 month-follow-up, all 105 participants will be contacted by email and invited to complete a semi-structured interview about their perceptions, attitudes, and experiences of the intervention. The first 25 to respond will be asked to complete the interview. The interview sample will include participants who dropped out of the intervention.

The trial also hopes to establish the potential for change and to establish initial estimates of the effectiveness of these online interventions for students in terms of within-group effect sizes associated with changes in depression, anxiety, and stress from baseline to 8 week and 3 months follow-ups. These data will be used to estimate the sample size for a future trial to ensure that the study is sufficiently powered. A conservative estimate using the 90 % upper confidence limit will be used to inform the sample size calculation of the future RCT.

Method/design

Study design

The study is an open trial of internet-delivered CBT treatment of depression, anxiety, and stress. After completing screening questionnaires, participants will be able to choose one of three online CBT interventions (Space from Depression, Space from Anxiety, and Space from Stress). The study protocol was approved by the University of Minnesota Institutional Review Board (March 18, 2015) (code number: 1503S64741).

Recruitment procedure

Participants will be recruited through three units at the University of Minnesota-Twin Cities, a public research university located in Minneapolis and St. Paul, Minnesota. The first unit is Student Counseling Services (SCS) which provides free short-term counseling to undergraduate and graduate students. The second is Boynton Mental Health Clinic (BMHC) which also provides short-term psychological services to full-time students at the University of Minnesota. Unlike SCS, BMHC also has psychiatrists on staff and is affiliated with the university health service. Finally, we will recruit participants through the International Student and Scholar Services (ISSS) office which provides counseling to foreign national students on campus.

The SilverCloud programs will be advertised to students through the three participating offices on campus via staff referrals, brochures, flyers, email announcements, and web sites and through the general student mental health web site. Regardless of how they learn about the programs, students can sign up online to get access to the programs. If a student expresses interest, they will be emailed a link to a web site with more information about the programs and the study. Interested students will sign the consent form and complete screening measures online and will be given feedback on their scores. This feedback consists of informing students of their overall score for each measure and whether their score indicates mild, moderate or severe depression, anxiety, or stress. This will allow them to make an informed decision on choosing one of the three programs. They will then be referred to a supporter from within the service where they learned about the study. Participants’ post outcome measures will be gathered 8 weeks after their initial log in or activation of the program and 3 months later (5 months after initial log in).

A second recruitment phase will be conducted at 3-month follow-up. All 105 participants will be contacted by email and invited to complete a semi-structured interview about their perceptions, attitudes, and experiences of the intervention. From those that respond expressing an interest, the first 25 will be selected to complete a semi-structured interview.

Sample size

A sample size of 35 participants per group is the standard expectation for feasibility studies [52]. This sample size will allow us to estimate the standard deviation of the symptom outcome measures (for the proposed RCT). We thus aim to recruit 105 participants in total across the three trial arms (see Fig. 1). In addition, a sample of between 10 and 15 people would be suitable to reach saturation through analysis of data collected in semi-structured interviews [16].

Fig. 1
figure 1

Flowchart of the study CONSORT

Eligibility criteria

All registered students at the University of Minnesota will be potentially eligible to participate. Eligibility criteria include being at least 18 years of age and self-reporting mild to severe symptoms of depression (5–27+) on the Patient Health Questionnaire-9 (PHQ-9; [26, 50]), mild to severe symptoms of anxiety (5–21+) on the Generalized Anxiety Disorder (GAD-7; [49]), or mild to severe symptoms of stress (15–34+) on the stress subscale of the Depression, Anxiety and Stress scale-21 (DASS-21; [28]). Students who score below the cutoff score on a given measure may still use the program module related to that measure, although their data will not be included in the analyses. Participants attending face-to-face individual or group counseling will be excluded. This includes all types of psychological therapy such as CBT. At baseline and 3 month assessments, participants will be asked if they have accessed any other form of psychological intervention throughout the time frame of the trial.

Students who score in the “red zone” in terms of risk of self-harm on the screening questions routinely used at SCS (i.e., the Behavioral Health Measure; [6]) will not be referred to the study. Students who score greater than 0 on the PHQ-9 self-harm item in the online screening will be automatically alerted that a counselor will try to reach them. They will be recommended to seek help from BMHC or UCCS. They will be telephoned within 1 working day and contacted by email if they cannot be reached by phone. There is also a “Help” button available on the site which can be accessed at any time and which directs students to a list of counseling resources and health services on campus.

Interventions

Internet-delivered cognitive-behavior therapy (iCBT) programs

Space from Depression, Space from Anxiety, and Space from Stress are eight-module online CBT interventions, delivered on a Web 2.0 platform using media-rich interactive content (see Table 1 for a description of the Space from Depression program). Specifically, Space from Depression focuses on cognitive and behavioral components including self-monitoring and thought recording, behavioral activation, cognitive restructuring, and challenging core beliefs. Space from Anxiety includes problem solving strategies, collaborative empiricism, empowerment, and mindfulness. Space from Stress is primarily aimed at boosting positive well-being, addressing stress in university settings, and encouraging users to build a more balanced, enjoyable, and meaningful life. The content of the program modules follows evidence-based CBT principles. Each program is structured and incorporates introductory quizzes, videos, informational content, and interactive activities, as well as homework suggestions and summaries. In addition, personal stories and accounts from other users are presented. Participants will be encouraged to complete one module per week over 8 weeks [32], with weekly reviews provided by their supporter (10–15 min).

Table 1 Overview of Space from Anxiety program

SilverCloud platform

The SilverCloud platform was developed in Ireland by a multi-disciplinary team of clinical, design, and development professionals under the company name of SilverCloud Health. Delivered through the SilverCloud platform, the programs for the treatment of depression, anxiety, and stress employ several innovative engagement strategies for improving the user experience. These are divided into several categories: personal, interactive, supportive, and social.

Personal

Users have their own secure homepage and can fill in a profile with basic information about themselves. Along with establishing a sense of ownership, this information allows the supporter to provide more personal feedback. The homepage is intended to provide a reflective space; users can document their thoughts and feelings, and these can be elaborated on within the journal application, which also acts as the vehicle for therapeutic writing exercises. Users have actions suggested to them, and as they complete program modules, their achievements are noted. Users are free to access the modules in any order they wish, contributing to a sense of empowerment. Along with the central content, a range of satellite applications are provided, such as a goal-setting application that can be used independently of the program content. Applications are released as the user completes the modules, with the intention of maintaining engagement by introducing new features over time and not overwhelming the user initially.

Interactive

The program includes a number of interactive elements and graphical exercises aimed at engaging users with the therapeutic content (e.g., reflecting on their own thinking style). Users also have the ability to comment on content within the programs. Both exercises and comments can be explicitly shared with the supporter. The user is provided with immediate feedback wherever possible; for example, when a mood chart is completed, the application item is graphically updated on the home page.

Supportive

All users have an assigned supporter who provides weekly reviews of their progress in the program. This support is asynchronous; the supporter sets a date to review a user’s progress and does not provide feedback, support or contact outside this time. The supporter can support multiple users and can review the work of all their users within an allocated time period. Such asynchronous online contact may be logistically easier to implement for many services compared to, for example, motivational interviewing and telephone support. Users are encouraged to share their content (such as completed exercises and comments) with their supporter. This enables the supporter to respond in a more personal way and provide guidance and encouragement to keep using the program. Adherence information is also available to the supporters so they can keep track of the users’ progress. Because this information is personally sensitive, users can see the supporter’s view of their data. Making the visibility of user data to the supporter more transparent, and giving the user the ability to change the sharing status of data, provides the user with a greater sense of control.

Social

As one step to making the programs social, the user can see anonymous indications of other people in the system. The intention is to reassure users that they are not alone in experiencing difficulties and that many other people have experienced similar problems and overcome them. Users can respond to content by indicating that they “like” it, and can see how many other people liked it, helping to reduce the sense of isolation. Introducing contact with other users within any online system raises a number of ethical concerns regarding the possibility for unhelpful or negative content or communications. Thus, other more detailed shared content (such as tips and ideas) is subject to supporter moderation.

Supporter’s role during treatment

Each participant will be assigned a supporter who will monitor the participant’s progress throughout the trial. Supporters will be professional staff at SCS, BMHC, or ISSS. Once participants choose a program, there will be a message from their supporter at their first login. This message welcomes them to the program, highlights aspects of the program, and encourages them in the use of the program. Each week, the supporters will log in and review participants’ progress, leaving feedback for them and responding to the work they have completed. The basic share level allows supporters to view users’ goals for the week, key messages, and progress points. If users wish to share more with their supporter, they can share journal entries. Participating supporters will receive training in the program and how to deliver feedback. Each supporter will provide post-session feedback of between 10 and 15 min per participant per session for their assigned participants. No adherence measure was developed to control for variance in the feedback provided as this natural variance between supporters reflects the expected variance in feedback found in a naturalistic setting.

Measures

Acceptability data will include information acquired through the online SilverCloud system including data on usage and engagement. In addition, the satisfaction with treatment (SAT) measure will be completed at post-treatment, and finally, data from semi-structured interviews with participants will inform acceptability.

The Patient Health Questionnaire-9 (PHQ-9), the Generalized Anxiety Disorder-7 (GAD-7), the stress subscale of Depression, Anxiety, Stress Scale-21 (DASS-21), and a sociodemographic and history questionnaire will be completed at screening. The following measures, PHQ-9, GAD-7, and stress subscale of DASS-21 will be completed again 8 weeks after their initial log in or activation of the program and 3 months later. All data will be obtained through online self-reported measures.

Sociodemographic information

The sociodemographic information and history questionnaire is based on the history questionnaire used in an earlier study [43]. It will be developed for the present study and will collect basic demographic information on the participants (e.g., age and gender) as well as information on any previous diagnosis of depression, anxiety, and stress disorders and on the length of time those symptoms were experienced; participants’ experiences of counseling, therapy, and medication for depression, anxiety, and stress; previous diagnosis of an organic mental health disorder such as schizophrenia, psychosis, or bipolar disorder; and items related to the presence of psychosis, alcohol and drug misuse, and/or any recent medical diagnosis.

Symptom outcome measures

The PHQ-9 [26, 50] is a 9-item self-report measure of depression that has been widely used in screening, primary care, and research. The PHQ-9 items reflect the diagnostic criteria for depression outlined in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition–Text Revision (DSM–IV–TR) [3]. Items are rated on a 0 to 3 scale. Summed scores range from 0 to 27; higher scores reflect a greater severity of depressive symptoms [26, 50].

The GAD-7 [49] comprises 7 items measuring symptoms and severity of GAD based on the DSM-IV diagnostic criteria for GAD. The GAD-7 has good internal consistency (.89) and good convergent validity with other anxiety scales. Items are rated on a 0 to 3 scale. Summed scores range from 0 to 21; higher scores indicate greater severity of symptoms [49].

Stress subscale of Depression, Anxiety and Stress scale-21 (DASS-21). The DASS-21 [28] is an abbreviated version of the original 42-item DASS. It is composed of three 7-item subscales measuring symptoms of depression, anxiety, and stress. Participants will be asked to rate the degree to which they endorse each of the seven stress items on a 4-point scale (0 to 3). Summed scores range from 0 to 21; higher scores indicate greater severity of symptoms. Scores on the DASS-21 have been found to be valid measures of depression, anxiety, and stress in non-clinical samples [17].

Acceptability

Acceptability of the intervention will be assessed using data acquired through the online SilverCloud system on usage and engagement with the intervention, including percentage of participants completing each module, average number of log ins, average time spent per session, and total time spent on the program.

We will also gather descriptive data on the feasibility of the recruitment process, such as numbers who expressed interest in participating, numbers who received links to the programs, and, of those, how many completed screening measures and how many used the program. Additionally, we will gather data on numbers recruited at each site and numbers of students selecting each of the three programs. These data will help to develop realistic estimates of enrollment accrual and attrition rates.

Satisfaction with treatment

The satisfaction with treatment (SAT) questionnaire [40] includes nine quantitative questions regarding satisfaction with accessing treatment online (e.g., How did this online treatment compare to previous treatments? 0 = “much better” to 4 = “not at all good”). The satisfaction measure also contains two qualitative questions asking participants to describe what they most liked and least liked about the online treatment.

Semi-structured interviews

Semi-structured interviews will include a schedule of questions derived from the literature that will explore the participants’ perceptions, attitudes, and experiences of the intervention. Interviews will be conducted by doctoral students via telephone and will last no more than 60 min.

Ethical considerations

Information made available to all prospective participants will inform them of exactly what is involved in participating, including the objectives of the trial. Participants will know that their involvement is voluntary and they can withdraw their participation at any time without prejudice. Informed consent will be collected online via checking a box indicating that they consent to participate.

Planned statistical analysis

Acceptability data in terms of usage and engagement will be assessed by analyzing the percentage of participants completing each module, the average number of logins, the average time spent per session, and total time spent on each of the three programs. As mentioned, feasibility will be assessed by analyzing the number of participants recruited at each site and the number of students selecting each of the three programs overall and at each site.

Satisfaction data will be analyzed using the descriptive and interpretative framework described by Elliott and Timulak [13].

Descriptive data will be presented from the semi-structured interviews. The transcriptions will be analyzed thematically using a pre-determined framework derived from the interview schedule and adapted and revised based on participant responses [45].

The outcome data will be treated as preliminary and interpreted with caution. Statistical analysis will be used to describe the characteristics of the sample and to provide initial estimates of parameters needed to inform the design of a future RCT. To assess whether change can be seen in the three programs, within group effect sizes (Cohen’s d) assessing change from baseline to 8 weeks and 3 months will be calculated. For Cohen’s d, an effect size of 0.2 to 0.3 can be considered a small effect, around 0.5 a medium effect, and 0.8 upwards a large effect [7]. A repeated measure ANOVA will be used to analyze the differences in the symptom outcomes across time points. An estimation of recovery will be made by identifying the number of participants completing each program who demonstrated a reduction of 50 % from pre-treatment PHQ-9, GAD-7, and DASS-21 stress subscale scores [39]. In addition, we will calculate the number of reliably changed and recovered participants using Jacobson and Truax’s [20] criteria.

Discussion

This study seeks to assess the feasibility, acceptability, and proof of concept of internet-delivered treatment for depression, anxiety, and stress for university students. The study will contribute to understanding how low-intensity internet-delivered programs can be integrated into service delivery in university settings. Thus, the study will not only contribute to the already established work on online psychological treatments worldwide but will extend the limited research assessing the effectiveness of online psychological treatments in realistic service delivery settings including existing empirical evidence for SilverCloud programs [4042]. Because participants will be recruited through three units at the University of Minnesota, including foreign national students from the International Student and Scholar Services, it is anticipated that the study will provide the researchers and service personnel with unique insight into students’ interest in and engagement with the programs, their user experience, and their satisfaction with the online delivery format. Any change in outcomes will be treated as preliminary and will be used to inform the methods for a future randomized trial of efficacy.

The symptom outcome measures (PHQ-9, GAD-7, DASS-21 stress subscale) that will assess depression, anxiety, and stress are well-established measures and each has been used in previous trials of internet-delivered and face-to-face treatments [42]. The acceptability and satisfaction measures will contribute information regarding what participants find satisfying with online treatments [40, 41]. Feasibility data will inform the procedures of a future trial (e.g., which programs to offer).

Trial status

Participant recruitment commenced in October 2015 and is ongoing. Numbers to date are 63.

References

  1. Alegria M, Kessler RC, Bijl R, Lin E, Heeringa SG, Takeuchi DT, et al. Comparing mental health service use data across countries. In: Andrews G, editor. Unmet need in mental health service delivery. Cambridge: Cambridge University Press; 2000. p. 97–118.

    Google Scholar 

  2. American College Health Association. National college health assessment reference group executive summary. Hanover: American College Health Association; 2010.

    Google Scholar 

  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-IV-TR. Washington: American Psychiatric Association; 2000.

    Google Scholar 

  4. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205. doi:10.1080/16506070903318960.

    Article  PubMed  Google Scholar 

  5. Association for University and College Counseling Center Directors. The association for university and college counseling center directors annual survey. Aurora: Association for University and College Counseling Center Directors; 2014.

    Google Scholar 

  6. Bryan C, Blount T, Kanzler K, Morrow C, Corso K, Corso M, Ray-Sannerud, B. Reliability and normative data for the Behavioral Health Measure (BHM) in primary care behavioral health settings. Fam Syst Health. 2014;32:89–100.

    Article  PubMed  Google Scholar 

  7. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Lawrence Earlbaum Associates; 1988.

    Google Scholar 

  8. Cuijpers P, Smit F, Oostenbrink J, de Graaf R, ten Have M, Beekman A. Economic costs of minor depression: a population-based study. Acta Psychiatr Scand. 2007;115(3):229–36. doi:10.1111/j.1600-0447.2006.00851.x.

    Article  CAS  PubMed  Google Scholar 

  9. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res. 2014;16:e130.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Day V, McGrath PJ, Wojtowicz M. Internet-based guided self-help for university students with anxiety, depression and stress: a randomized controlled clinical trial. Behav Res Ther. 2013;51(7):344–51. doi:10.1016/j.brat.2013.03.003.

    Article  PubMed  Google Scholar 

  11. Donohue EO, Fisher JE, editors. Cognitive Behavior Therapy: Core principles and practice. Hoboken: Wiley; 2012.

    Google Scholar 

  12. Dyson R, Renk K. Freshmen adaptation to university life: depressive symptoms, stress, and coping. J Clin Psychol. 2006;62:1231–44.

    Article  PubMed  Google Scholar 

  13. Elliott R, Timulak L. Descriptive and interpretative approaches to qualitative research. In: Miles J, Gilbert P, editors. A handbook of research methods in clinical and health psychology. Oxford: Oxford University Press; 2005. p. 147–60.

    Google Scholar 

  14. Farrer L, Gulliver A, Chan JK, Batterham PJ, Reynolds J, Calear A, Griffiths, KM. Technology-based interventions for mental health in tertiary students: systematic review. J Med Internet Res. 2013;15:e101.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Gruttadaro N, Crudo D. (2012). College students speak: A survey report on mental health. Retrieved from National Alliance on Mental Illness website: https://www.nami.org/About-NAMI/Publications-Reports/Survey-Reports/College-Students-Speak_A-Survey-Report-on-Mental-H.pdf.

  16. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59–82. doi:10.1177/1525822X05279903.

    Article  Google Scholar 

  17. Henry JD, Crawford JR. The sort-form version of the Depression Anxiety Stress Scales (DASS-21): construct validity and normative data in a large nonclinical sample. Br J Clin Psychol. 2004;44:227–39.

    Article  Google Scholar 

  18. Hersen M, Sturmey P, editors. Handbook of evidence-based practice in clinical psychology, volume 2, adult disorders (Vol. 2). Hoboken: Wiley; 2012.

    Google Scholar 

  19. Hirai R, Frazier P, Syed M. Psychological and sociocultural adjustment of international students: trajectories and predictors. J Couns Psychol. 2015;62:438–52. http://dx.doi.org/10.1037/cou0000085.

    Article  PubMed  Google Scholar 

  20. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59:12–9.

    Article  CAS  PubMed  Google Scholar 

  21. Johnson EM, Coles ME. Failure and delay in treatment-seeking across anxiety disorders. Community Mental Health Journal. 2013;49:668–574.

    Article  PubMed  Google Scholar 

  22. Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ormel J, Wang, PS. The global burden of mental disorders: an update from the WHO world mental health (WMH) surveys. Epidemiol Psichiatr Soc. 2009;18(1):23–33. doi:10.1017/S1121189X00001421.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593–602. doi:10.1001/archpsyc.62.6.593.

    Article  PubMed  Google Scholar 

  24. Kitzrow MA. The mental health needs of today’s college students: challenges and recommendations. NASPA J. 2003;41:167–81.

    Article  Google Scholar 

  25. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ. 2004;82:858–66.

    PubMed  PubMed Central  Google Scholar 

  26. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9 validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–13.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Lee C, Dickson D, Conley C, Holmbeck G. A closer look at self-esteem, perceived social support, and coping strategy: a prospective study of depressive symptomatology across the transition to college. J Soc Clin Psychol. 2014;33(6):560–85. doi:10.1521/jscp.2014.33.6.560.

    Article  Google Scholar 

  28. Lovibond SH, Lovibond PF. Manual for the depression anxiety stress scales. 2nd ed. Sydney: Psychology Foundation; 1995. ISBN 7334-1423-0.

    Google Scholar 

  29. Mohr DC, Ho J, Duffecy J, Baron KG, Lehman KA, Jin L, Reifler, D. Perceived barriers to psychological treatments and their relationship to depression. J Clin Psychol. 2010;66:394–409.

    PubMed  PubMed Central  Google Scholar 

  30. Nathan PE, Gorman JM, editors. A guide to treatments that work (treatments that work). 3rd ed. New York: Oxford University Press, USA; 2007.

    Google Scholar 

  31. National Institute for Health and Clinical Excellence. Computerised cognitive behaviour therapy for depression and anxiety. Technology Appraisal 97. London: National Institute for Health and Clinical Excellence; 2006.

    Google Scholar 

  32. National Institute for Health and Clinical Excellence. The treatment and management of depression in adults. London: National Institute for Clinical Excellence; 2009. p. 64.

    Google Scholar 

  33. National Institute for Health and Clinical Excellence (NICE). Commissioning stepped care for people with common mental health disorders. London: National Institute for Health and Clinical Excellence; 2011.

    Google Scholar 

  34. Newton-Taylor B, Adlaf E, Gliksman L, Demers A. The prevalence of elevated psychological distress among Canadian undergraduates: findings from the 1998 Canadian campus survey. J Am Coll Heal. 2001;50(2):67–72. doi:10.1080/07448480109596009.

    Article  Google Scholar 

  35. Rapaport MH, Clary C, Fayyad R, Endicott J. Quality-of-life impairment in depressive and anxiety disorders. Am J Psychiatry. 2005;162(6):1171–8. doi:10.1176/appi.ajp.162.6.1171.

    Article  PubMed  Google Scholar 

  36. Reynolds III CF, Beekman ATF, Sijbrandij EM, Cuijpers P, Koole SL, Andersson G. Adding psychotherapy to antidepressant medication in depression and anxiety disorders: a meta-analysis. World Psychiatry. 2014;13(1):56–67. doi:10.1002/wps.20089. Filosofiska fakulteten.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Richards D. Prevalence and clinical course of depression: a review. Clin Psychol Rev. 2011;31(7):1117–25. doi:10.1016/j.cpr.2011.07.004.

    Article  PubMed  Google Scholar 

  38. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329–42. doi:10.1016/j.cpr.2012.02.004.

    Article  PubMed  Google Scholar 

  39. Richards DA, Suckling R. Improving access to psychological therapies: phase IV prospective cohort study. Br J Clin Psychol. 2009;48:377–96.

    Article  PubMed  Google Scholar 

  40. Richards D, Timulak L. Satisfaction with therapist-delivered vs. self-administered online cognitive behavioural treatments for depression symptoms in college students. Br J Guid Couns. 2012;41(2):193–207. doi:10.1080/03069885.2012.726347.

    Article  Google Scholar 

  41. Richards D, Timulak L. Client-identified helpful and hindering events in therapist delivered vs. self-administered online cognitive-behavioural treatments for depression in college students. Couns Psychol Q. 2012;25:251–62.

    Article  Google Scholar 

  42. Richards D, Timulak L, Doherty G, Sharry J, Bligh J, Colla A. Internet-delivered treatment for depression: its potential as a low-intensity community intervention for adults with depression: Protocol for a randomized controlled trial. BMC Psychiatry. 2014;14(147):1–11. doi:10.1186/1471-244X-14-147.

    Google Scholar 

  43. Richards D, Timulak L, Hevey D. A comparison of two online cognitive-behavioural interventions for symptoms of depression in a student population: the role of therapist responsiveness. Couns Psychother Res. 2013;13(3):184–93. doi:10.1080/14733145.2012.733715.

    Article  Google Scholar 

  44. Richardson K, Rothstein H. Effects of occupational stress management intervention programs: a meta-analysis. J Occup Health Psychol. 2008;13:69–93.

    Article  PubMed  Google Scholar 

  45. Ritchie J, Lewis J. Qualitative research practice: a guide for social science students and researchers. London: Sage Publications; 2003.

    Google Scholar 

  46. Royal College of Psychiatrists. The mental health of students in higher education. London: Royal College of Psychiatrists; 2003.

    Google Scholar 

  47. Royal College of Psychiatrists. Mental health of students in higher education. London: Royal College of Psychiatrists; 2011. p. 97.

    Google Scholar 

  48. Rutter M, Sroufe LA. Developmental psychopathology: concepts and challenges. Dev Psychopathol. 2000;12(3):265–93.

    Article  CAS  PubMed  Google Scholar 

  49. Spitzer RL, Kroenke K, Williams JW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7. doi:10.1001/archinte.166.10.1092.

    Article  PubMed  Google Scholar 

  50. Spitzer R, Kroenke K, Williams J. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. J Am Med Assoc. 1999;282:1737–44.

    Article  CAS  Google Scholar 

  51. Substance Abuse and Mental Health Services Administration. (2009). Results from the 2009 national survey on drug use and health: Volume I. Summary of national findings. Retrieved from http://archive.samhsa.gov/data/NSDUH/2k9NSDUH/2k9Results.htm.

  52. Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;3(15):264.

    Article  Google Scholar 

  53. Wang PS, Berglund PA, Olfson M, Kessler RC. Delays in initial treatment contact after first onset of a mental disorder. Health Serv Res. 2004;39(2):393–416.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The trial will be jointly funded from monies and resources provided by SilverCloud Health Ltd. and University of Minnesota.

Authors’ contributions

PF and DR are the principal investigators for the project. PF, DR, and JM conceptualized the initial trial design and wrote the draft manuscript. All authors reviewed and made revisions and suggestions to the manuscript. All authors read and approved the final manuscript.

Competing interests

Derek Richards is Director of Clinical Research and Innovation at SilverCloud Health. Jacqueline Mooney and Christopher Bonner are both assistant psychologists at SilverCloud Health. Derek Richards is also Research Fellow, School of Psychology, Trinity College Dublin (TCD) and adheres to both research ethics and TCD policy on good research practice. The other authors declare no conflict of interest related to this study.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Derek Richards.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frazier, P., Richards, D., Mooney, J. et al. Acceptability and proof of concept of internet-delivered treatment for depression, anxiety, and stress in university students: protocol for an open feasibility trial. Pilot Feasibility Stud 2, 28 (2016). https://doi.org/10.1186/s40814-016-0068-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40814-016-0068-9

Keywords