Table 1 Data collection and evaluation plan
Objective | Indicators and outcome measures | Data collection method | Measure of success |
|---|---|---|---|
Feasibility Checklist | |||
Feasibility | Recruitment rate: % of participants recruited/time | Recruitment report by the Teagasc. | Recruitment: Successfully recruit a minimum two farmers from each of the four farm types: dairy, beef, sheep and tillage |
Retention and follow-up rates: % of participants completed the stage 1, stage 2, stage 3, stage 4 (Fig. 1) | Audio recording of the in-person sessions Intervention checklist (Additional file 1) | ||
Time required to recruit target sample size: 16 participants recruited/time | Recruitment report by the Teagasc | ||
Adherence rates: % of participants completed the stage 1, stage 2, stage 3, stage 4 | Field notes and Memos from the in-person sessions | Retention rate: 80% of the participants complete the in-person session & A minimum of 12 participants complete follow-up interviews | |
Rate of completion of the intervention: % of intervention activities completed by the participants | Field notes and Memos from the in-person sessions | ||
SMS Survey | |||
Representation of farm population: % of participants recruited from four farm types/total participants | Pre-intervention interview data | ||
Barriers & facilitators to set up the in-person event: Factors identified by the facilitators | Recruitment report by the Teagasc Audio recording of the in-person sessions Field notes from the in-person sessions | Programme structure: At least 80% of the participant rate the programme structure and content as satisfactory | |
Is the structure of the programme (length of the event, structure and content of the intervention) realistic, clear and reasonable for the participants?: Average score for each of the responses in the exit survey | Exit survey | ||
Post-intervention interview | |||
The ability of the participants to carry out the target behaviours (safety behaviours) addressed in the study: Participants’ feedback about their experience | Safety training procedure Exit survey Post-intervention interview | ||
Fidelity Checklist (1) | |||
Fidelity: Intervention Design | 1—Provide information about treatment dose in the intervention condition: Detailed information about the intervention is presented in the intervention checklist, such as: a. Length of session b. Number of sessions c. Content of each session d. Duration of contact over time | Intervention checklist | Intervention report: Detailed report on the development of the intervention, content of the intervention and BCTs included in it using TiDier checklist |
2 Theoretical model upon which the intervention is based is clearly articulated in the methodology paper and includes the following information: a. The active ingredients are specified and incorporated into the intervention b. Use of experts or protocol review group to determine whether the intervention protocol reflects the underlying theoretical model or clinical guidelines c. Plan to ensure that the measures reflect the hypothesised theoretical constructs/mechanisms of action | A manuscript describing the development of BeSafe Intervention | Trial registration: Register the study on ISRCTN registry | |
3 Potential confounders that limit the ability to make conclusions at the end of the trial are identified? | Post-intervention interview | ||
4 Plan to address possible setbacks in implementation (i.e. back-up systems or providers) | Risk assessment document | ||
Fidelity: Treatment providers* | 1 Description of how providers will be trained (manual of training procedures) | Intervention manual | Intervention manual: Complete and upload the intervention manual in a public repository |
Fidelity: Treatment delivery | 1 Method to ensure that the content of the intervention is delivered as specified (content, dose, process): Number of intervention components delivered as intended/Number of intervention components expected to be delivered | Audio recording Intervention checklist Field notes | Delivery: A minimum of 80% of the intervention components presented in the intervention checklist are delivered. |
2 Assessment of non-specific treatment effects: Participants report on the experience of the programme vs expected outcome | Post-intervention interview | ||
5 Use of Intervention manual | Project information sheet | ||
6 There is a plan for the assessment of whether or not the active ingredients were delivered: Number of active ingredients delivered as intended/Number of active ingredients present in the BeSafe intervention | Audio recording Intervention checklist Field notes | ||
7 There is a plan for the assessment of whether or not proscribed components were delivered. (e.g. components that are unnecessary or unhelpful) | Audio recording Intervention checklist Field notes | ||
8 There is a plan for how will contamination between conditions be prevented | N/A | ||
9 There is an a priori specification of treatment fidelity (e.g. providers adhere to delivering > 80% of components): Reference to ‘measure of success’ column | Data collection and evaluation plan | ||
Fidelity: Receipt of Treatment | 1 There is an assessment of the degree to which participants understood the intervention: Participants’ report on their understanding of the intervention skills | Safety training procedure Exit poll Post-intervention interview | Receipt: A minimum of 60% of the participants complete the target behaviours at home |
2 There are specifications of strategies that will be used to improve participant comprehension of the intervention: Completion of tailored plan using the safety training procedure document an participants’ feedback | Safety training procedure Exit poll Post-intervention interview | ||
3 The participants’ ability to perform the intervention skills will be assessed during the intervention period: Self-report on the confidence in completing the target behaviours and participants’ feedback | Safety training procedure Exit poll Post-intervention interview | ||
4 A strategy will be used to improve subject performance of intervention skills during the intervention period: Hands-on practice session on demonstration, completion of tailored plan using the safety training procedure and participants’ feedback | Peer to peer demo Safety training procedure | ||
5 Multicultural factors considered in the development and delivery of the intervention (e.g. provided in native language; protocol is consistent with the values of the target group) | N/A | ||
Fidelity: Enactment of Treatment Skills | 1 Participant performance of the intervention skills will be assessed in settings in which the intervention might be applied: Participants’ self-report on the completion rate of safety goals (target behaviours) | SMS Survey Post-intervention interview | |
2 A strategy will be used to assess performance of the intervention skills in settings in which the intervention might be applied: Participants’ reported experience on completing the safety goals at home | Post-intervention interview | ||
Acceptability checklist | |||
Retrospective acceptability | 1 Affective attitude (How did the participant feel about the programme): Average satisfaction score for the programme in the exit survey | Safety training procedure Exit survey Follow-up evaluation interview | A minimum of 80% of the participants rate the programme on exit survey as “Agree” or “Strongly agree” A minimum of 80% of the participants rate their confidence in completing safety goals at least 7 out of 10 |
2 Burden (What did the participants say about the structure and ease to understand): Average satisfaction score for the programme structure in the exit survey and participants’ feedback | |||
3 Ethicality (To what extent the strategy helped in performing the farm tasks): Average satisfaction score for the programme and topics covered in the programme in the exit survey | |||
4 Intervention coherence (How did the participant feel in terms of understanding the tasks and performing them by himself): Participants’ report on what they learned during the programme | Evidence for perceived benefits and effectiveness of the intervention in the follow up interview. | ||
5 Opportunity costs (What were the benefits the participants perceived): Participants’ feedback | |||
6 Perceived effectiveness (To what extent did the participant feel that strategy was effective): Participants’ report on what they learned during the programme | |||
7.Self-efficacy (How confident and comfortable was the participant at performing the task): Average confidence score for each of the target behaviours in the safety training procedure document | |||