Table 1 Measures by variable/construct, measure characteristics, timepoint, method, informant, and time burden
Variable/construct | Measure | Measure characteristics | Timepoint | Method | Informant | Time |
|---|---|---|---|---|---|---|
Aim 1 | ||||||
Barriers and facilitators | Interview guide # 1 | A semi-structured qualitative interview will be conducted with BHS to elicit views about perceived barriers and facilitators to participation in consultation sessions and conducting groups with students (e.g., What would make it difficult for you to participate in consultation sessions and conduct groups with students? Now, please tell me what would make it easier for you to participate in remote training, receive consultation remotely or conduct groups with students?) | Pre-trial | Coding | Behavioral health staff | 20 min |
Aim 2 | ||||||
Assess prototype | Interview guide # 2 | The interview includes a description of the first platform prototype; it will describe each training and consultation component. BHS will be asked whether the different components of the training and consultation and group implementation would be feasible (e.g., We’re interested in your thoughts about how feasible it would be to use remote video technology in your school) and acceptable (e.g., We’re interested in your thoughts about how acceptable the remote video technology is) using a 5-point scale [46]. They will also be asked why the component is or is not feasible/acceptable; whether they would be willing to participate in remote consultation. | Pre-trial | Coding | Behavioral health staff | 30 min |
Assess prototypes | Surveys # 1–3 | The surveys include the Intervention Appropriateness Measure [IAM], the Acceptability of Intervention Measure [AIM], and the Feasibility of Intervention Measure [FIM] [40]. The measures are comprised of 4 items, each rated on a 5-point scale (1=completely disagree to 5=completely agree). Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α’s from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions [40]. Survey # 3 will also include the Intervention Usability Scale (IUS), an adaptation of the System Usability Scale (SUS) [41]. The SUS is a widely used scale for assessing usability of digital products. We adapted the SUS for the evaluation of instructional videos. Respondents will also be asked to provide comments to explain their answers (e.g., “please comment on the video about using remote consultation technology”). | Pre-trial | Rating scale and coding | Behavioral health staff | 30 min |
Pre-trial activities | ||||||
Tier 2 screening | Strengths and Difficulties Questionnaire (SDQ) [46] with Impact Supplement [47] | The SDQ is a 25-item, 3-point scale (0 = not true; 2 = certainly true) questionnaire used to assess the psychological adjustment of children and youth, ages 4–17. | Pre-treatment | Rating scale | Parents/teachers | 5 min |
Aim 3: implementation trial | ||||||
Implementation measures | ||||||
Content fidelity of group CBT | Coping Power and CATS Content Fidelity Checklist (CFC) [48] | The CFC reflects each activity component of the session agenda of the treatment protocols. Raters use a yes/no response scale to indicate whether or not the implementer covered a particular component as captured in audio recordings of the group sessions. | Ongoing | Coding | Research staff | 40 min |
Content fidelity of CI/CO | The Check-In/Check-Out Fidelity Checklist is a 9-10-item checklist used by Tier 2 implementers during morning check-in and afternoon check-out rated as either occurring or not occurring. | Weekly | Coding | Research staff | 10 min | |
Adoption | Adoption Inventory (AI) | The AI is an Excel track sheet listing the number of times each intervention is used per school, per condition | Ongoing | Coding | Research staff | 1 min |
Dosage | Dosage Inventory (DI) | The DI is an Excel track sheet exported from the project website listing the number of times and length of time each video module is accessed by BHS in each condition | Ongoing | Digital | Research staff | 5 min |
Penetration | Penetration Inventory (PI) | The PI is an Excel track sheet listing EBP penetration at the student level (students receiving EBPs at Tier 2) | Ongoing | Coding | Research staff | 1 min |
Student outcome measures | ||||||
Mental health symptoms | Behavior Assessment System for Children - 3rd Edition (BASC-3) [52] | Parents will complete either the web-based or paper and pencil version of the BASC-3. The BASC-3 is a 138-item, 4-point, Likert-type (1=never, 2=sometimes, 3=often, 4=almost always) rating scale for assessing parental report of child mental health functioning, standardized for ages 2.5 to 18 years. The BASC-3 has excellent psychometric properties. The BASC-3 (Aggression, Conduct Problems, and Anxiety scales) will be used for children being considered for Tier 2 and administered at pre- and post-treatment for children who participate in CPP, CATS, or CICO. | Pre/post-treatment | Rating scale | Parent | 20 min |
Student academic engagement | Engagement versus Disaffection with Learning - Teacher Report (EvsD-Teacher) [53, 54]. | The EvsD will be completed by teachers for all students receiving Tier 2 interventions. This is a 20-item, four-point (1 = not at all true; 4 = very true) instrument with four sub-scales: (a) Behavioral Engagement, (b) Emotional Engagement, (c) Behavioral Disaffection, and (d) Emotional Disaffection. Internal consistency for students in grades 3–6 was .81–.87 for the four subscales. We will use the average score for each of the four scales at pre- and post-participation in CPP, CATS, or CICO. | Pre/Post-treatment | Rating scale | Teacher | 10 min |
Perception of training support | Qualitative Interview Guide # 3 | Semi-structured qualitative interviews are conducted with Tier 2 implementers and administrators in each condition to elicit views and perspectives about the perceived feasibility, acceptability, appropriateness, and usability of the training support they received. | Post-trial | Coding | Behavioral health staff/administrators | 30 min |