Table 3 Evaluation of methodological components of the Mini-AFTERc pilot study

1. Did the feasibility/pilot study allow a sample size calculation for the main trial?

We estimate a total sample size of 152 completing follow-up would be required for a main trial.

A sample size calculation (n=152 reaching follow-up) has been performed from the estimated effect size of 0.67. This would still be achievable with four cancer centres if they can identify approximately 400 patients each (Total eligible = 1689 patients).

2. What factors influenced eligibility and what proportion of those approached were eligible?

FCR4 cut-offs were the key factor in eligibility.

A greater than anticipated proportion of patient screened met FCR4 screening criteria (25% anticipated; 49% actual). One third of patients who were ineligible after screening scored just above the FCR4 cut-off (score of 15).

45 of 92 patients (49%) who completed screening were eligible based on FCR4 cut-offs.

32% or 15 of the 47 ineligible patient (FCR score out with 10-14) scored just above the cut-off (15 or 16).

Increasing the cut-off to include this group is theoretically justifiable and may boost recruitment potential significantly.

3. Was recruitment successful?

Overall recruitment targets of 50 patients in each study group was not achieved. Recruitment to control group was more successful than to intervention group.

Recruitment was impacted by:

• Variability in participating cancer centres’ follow-up procedures.

• Administrative restrictions in one centre.

• Systemic issues with SBCNs finding protected time to practice the intervention.

• National lockdown due to the Covid-19 pandemic.

28 of 51 patients screened at control centres were eligible and successfully recruited into the control group (56% of expected control sample of 50 patients).

17 of 41 patients screened at intervention centres were successfully recruited into the intervention group (34% of expected intervention sample of 50 patients).

49% of those screened were eligible (45 of 92 patients)

4. Did eligible participants consent?

All participants who were eligible after screening agreed to participate.

100% of those screened as eligible (45 patients; 49.4% of total screened) agreed to participate in the study.

5. Were participants successfully randomized and did randomization yield equality in groups?

Not applicable.

Not applicable.

6. Were blinding procedures adequate? 

Not applicable.

Not applicable.

7. Did participants adhere to the intervention?

Patient adherence to the intervention was not assessed because Mini-AFTERc does not require adherence to specific criteria or conditions.

A fidelity of intervention implementation measure has been developed to assess SBCN adherence to the Mini-AFTERc training manual.

The fidelity of implementation measure has preliminary evidence of good reliability:

Brandt, N. G., McHale, C. T., & Humphris, G. M. (2020). Development and Testing of a Novel Measure to Assess Fidelity of Implementation: Example of the Mini-AFTERc Intervention. Frontiers in psychology, 11, 601813. https://doi.org/10.3389/fpsyg.2020.601813

8. Was the intervention acceptable to the participants?

Both patients and nurses found the Mini-AFTERc intervention to be valuable and acceptable.

13 of 15 intervention patient participants completed CARE and modified-MISS questionnaires following the intervention discussion. Mean scores for all 10 care questions were between 4 and 5 (very good and excellent). Mean scores for modified MISS questions: patients neither agreed or disagreed with statements that the nurse had relieved their fear about the cancer returning and that they felt much better about the chances of the cancer not coming back. Patients disagreed with the statement that talking to the nurse had not helped at all.

Patient interviews (9/10 control patient): Patients perceived a general lack of structured psychosocial support from the cancer centre following the end of primary treatment. They thought that it was important and acceptable to provide FCR intervention for those who needed it.

SBCN interviews: SBCN valued the structured approach of the Mini-AFTERc intervention, it helped them with discussion time management, rose awareness of FCR, increased their confidence in talking about FCR. SBCN thought Mini-AFTERc was a more strcutred approach to the FCR discussion they had with patients regularly

9. Was it possible to calculate intervention costs and duration?

The cost of intervention implementation was not assessed but it was designed to be integrated into current cancer care practice by making use of existing staff and resources.

Each intervention discussion is structured to last a maximum of 30 minutes. Overall time would be determined by how many patients receive it in practice and/or whether it is only offered to certain patients.

SBCN interviews: Some SBCN did raise concerns about finding protected time within the current workload to conduct the Mini-AFTERc intervention.

10. Were outcome assessments completed?

Patient outcome assessments were well completed.

80% of follow-up questionnaire items were successfully competed by patient participants.

Missing follow-up items were ‘missing complete at random’ and there were no patterns to data missingness; χ² (602) = 370.62, p = 1.00).

11. Were outcomes measured those that were the most appropriate outcomes?

FCR4 questionnaires, which were the primary patient outcome measure, were well completed by patients.

The mobile phone application was not utilised very often, primarily due to Operating System restrictions (Apple iOS version not available). All patient were happy to complete pencil and paper version of questionnaires.

Missing data on completion of the FCR4 items was almost non-existent.  That is, of the patients who attempted completion of the 4 items it was found that all 4 items were responded to without items being missed out.

The Raykov's factor reliability coefficient for FCR4 was 0.913 (equivalent to Cronbach’s alpha).

Confirmatory factor analysis of FCR4 showed a ‘very close fit’ to a unidimensional latent variable.  Chi square equalled a low value of misfit (p>.3), indicating exceptional measurement clarity.

FCR4 scoring had little to no floor or ceiling effect. One percent scored at the lowest possible rating on the total score and two of the patients (2%) scored at the highest but one value on the measure.

12. Was retention to the study good?

Retention rate, defined as retuning the final (12 week) FCR4 follow-up questionnaire, was almost 80%.

Two patients dropped out before receiving the intervention. Both were recruited just prior to the national Covid-19 lockdown, when there was a lot of uncertainty about health service provision.

9 out of 45 patients did were lost to follow-up (20.1%).

13. Were the logistics of running a multicentre trial assessed?

Recruitment approaches varied by centre due to differences in follow-up practices, which influenced the success of recruitment at each centre. All centres did recruit patients. Administrative and organisational issues challenged recruitment (see response to Q3).

For this study, two research staff (supported by PIs) were able to co-ordinate and manage research activities at the four centres effectively.

Recruitment was less efficient at the centre where administrative support from the centre was restricted. The centre where recruitment was conducted entirely via telephone was as successful as other centres were recruitment was conducted in person.

14. Did all components of the protocol work together?

Training, screening, consent and follow-up protocols were successful, and all worked well together.

Intervention implementation was successful conducted by the SBCNs.

Patient interviews were successfully conducted but were heavily skewed towards control patients. Interviews were conducted during the first Covid-19 national lockdown in the UK (May-June 2020).

Recruitment protocols worked and patients were recruited, however recruitment was challenged by various factors, which reduced the efficiency of recruitment at some centres.

The SBCN training programme was successful and promptly delivered with positive feedback from SBCNs.

96% of patients were screening after providing informed consent and 100% agreed to participation if eligible after screening. Retention rate across the sample was 80%.

Required number of interviews were met with control patients (35% actual, 25% expected) but not with intervention patients (11 %actual, 25% expected).

All patient who were due to receive the Mini-AFTERc intervention did within two weeks of recruitment.