Table 1 Rationale for conducting a pilot/feasibility study
Metric and reason | Example of feasibility objectives in the literature | Summary of systematic review results | Result of feasibility outcome in Project GEMS [33] |
|---|---|---|---|
Process: assesses the feasibility of the processes that are key to the success of the main study | Determine recruitment rates [23, 24] (e.g. response of participants to recruitment strategies, proportion of respondents who remain interested in study after information and screening) | Recruitment via MS Societies in the prospective location, clinician referrals, and trial awareness (through posters and leaflets) [7, 41, 44–46, 48, 52, 53, 55] Rationale for non-recruitment: travel difficulty, time restrictions [37] | Overall recruitment rate, 52% Successive recruitment via postal invitation, 25%; email invitation, 30%; and telephone invitation, 22% |
Determine ease of randomisation [24] (e.g. willingness of participants to be randomised to the proposed treatment group(s) | No examples | Not assessed | |
Resources: assesses the time and resource problems that can occur during the main study | Estimate retention of participants in the study [23, 24]a (e.g. number of participants completing all aspects of study, number and reason for attrition) | All participants completed study [35, 37, 39, 41, 44, 47, 52] Drop out reasons: the intervention [7, 38, 58]; changes in time commitments [35, 51, 58]; unable to travel [42, 45, 46]; MS relapse [52]; other medical issues [7, 43, 45, 58]; non-compliance with the study protocol [7]; and lost contact [35, 37, 45] | 90% of all participants completed study Drop out reasons: changes in time commitment and other medical issues |
Demonstrate appropriate eligibility criteria [23, 24]a (e.g. are criteria too inclusive/exclusive) | All participants met inclusion criteria [47, 55] Exclusion reasons: participants were too active [35, 41]; too old [35]; recent relapse [35]; participation in another trial [35]; participation in formal rehabilitation [42]; non-MS diagnosis [35]; recent change in disease modifying therapy [35]; high fall history [41]; and cognitive deficits [41] | 32% of interested parties did not meet inclusion criteria. Exclusion reasons; too active, low self-reported disability level | |
Estimate barriers/refusals to participation [23, 24]a (e.g. participant transportation problems) | Barriers identified: unable to travel [45, 46]; a change in personal time commitments [35, 51, 58]; MS relapse [52]; and other medical issues [7, 43, 45, 58] | 10% of interested parties chose not to participate; unable to commit time | |
Demonstrate compliance with study protocol [23, 24]a (e.g. do participants adhere to correct dosage of intervention sessions) | Recorded via attendance at intervention [35, 37–40, 42–47, 50, 52, 58]; participant self-completed activity diaries [35, 37, 45–47] | 75% of intervention participants were fully compliant with exercise sessions | |
Demonstrate participants reaction to data collection and outcome assessments [23, 24]a (e.g. participants understanding of data collection tools) | Compliance problems identified: participants unable to complete walking tasks [57]; general difficulties with assessment procedures [37] | Time to complete outcome questionnaires: baseline, 40 min; follow-up 48 min | |
Estimate access to/cost of equipment, space, personnel time [24] (e.g. total cost of intervention delivery) | Cost identified: staff, equipment and facility overheads [54] | Cost per intervention participant: US$121.18 | |
Determine the suitability of the intervention in the proposed setting [24] (e.g. suitability of a home-based exercise programme) | Feedback questionnaires and telephone interviews indicate intervention suitable | ||
Determine clinician training needs and competence [24] (e.g. training in outcome assessment and/or intervention delivery) | Recorded via reliability of assessor [44], Reported as staff training requirements described [42, 44, 55] | Not assessed | |
Management: assesses potential human and data management problems | Estimate research site capacity [24] (e.g. phone-line, database, clinic/research site capacity) | Reported as staff time required for recruitment [49] | Staff preparation and reporting time: 263 h across 4 staff members |
Estimate equipment usage [23] (e.g. ease of availability, personnel time, establishment of backup plan if equipment unavailable/broken) | Problems identified: equipment related data collection problems [51] | Not assessed | |
Determine processing time for data collection [23, 24] (e.g. time to mail data collection materials, time to complete outcome assessment) | Reported asa staff time required for equipment processing and preparation [51] | Mail turn-around-time to receive outcome assessments: 3 weeks | |
Estimate data completeness [24] (e.g. missing data items, missing outcomes) | No examples | Missing data: 2.5% at baseline, 7.2% at follow-up | |
Estimate data entry [24] (e.g. erroneous data) | No examples | Staff time to enter and check data: 61 h | |
Determine software appropriateness for data [24] (e.g. capacity of software for data analysis) | Requirement identified: multiple software types necessary [55] | Not assessed | |
Estimate processes to ensure and/or audit treatment fidelity [24] (e.g. clinicians adherence to protocol) | Reported as experienced staff providing feedback on intervention delivery to the intervention instructors [42] | Not assessed | |
Scientific: assesses the safety, burden data collection and response to the study | Estimate challenges perceived/experienced by study personnel [23] (e.g. skills required to use assessment protocol) | Requirement for more time for participants to complete assessments [49] | Not assessed |
Determine data collection materials appropriateness [23] (e.g. user friendly for data collection personnel) | No examples | Participants commented the outcome assessments were burdensome | |
Demonstrate potential extraneous variables which may threaten the validity of the research [24] (e.g. participant’ prior knowledge of intervention content) | No examples | Not assessed | |
Determine the acceptability to participants of the intervention(s) [24] (e.g. participants view on intervention before/during/after) | Refer to original publications for individual details of the most acceptable exercise intervention [35, 37, 56, 57] | Positive written and verbal feedback from participants on the appropriateness of the intervention | |
Estimate data variability in controlled trials [23, 24] (e.g. statistical analysis performed to establish baseline differences between groups) | No significant baseline differences in demographic or clinical metrics | ||
Estimate treatment effect [23] (e.g. effect of primary outcome—not recommended) | Significant interaction—recommend primary outcome [37, 39, 45, 46, 52, 58] No significant interaction—cannot recommend primary outcome [44, 48, 49] | Significant interaction—recommend primary outcome | |
Determine appropriateness of target group for intervention [24] (e.g. are participants receptive to change expected in intervention) | Positive feedback from participants on the appropriateness of the intervention [56] | Positive written and verbal feedback from participants on the appropriateness of the intervention |