Important considerations for feasibility studies in physical activity research involving persons with multiple sclerosis: a scoping systematic review and case study

Multiple sclerosis (MS) is a disease of the central nervous system characterised by inflammation, axonal demyelination and transection and neurodegeneration. The damage and its location within the CNS [1, 2] manifest as a loss of physical and psychological function, worsening of symptoms and reduction in quality of life (QOL). Over the past 20 years, there have been an increasing number of randomised controlled trials (RCTs) of exercise as a therapeutic intervention for managing the consequences of MS. There is substantial evidence from these RCTs for short-term benefits of exercise and physical activity amongst persons with MS who have mild-to-moderate disability [3–5]. Exercise is now considered one of the most important interventions for inclusion in the management of MS and its consequences [6].

Importantly, there is ongoing debate about the strength of the existing research regarding exercise and MS (e.g. [7] vs. [8]), and this is associated with limitations of previous RCTs. For example, there is limited evidence for the duration of benefits from participation in the exercise intervention after cessation of the intervention period [9–11]. Some data suggest that the benefits of exercise are not retained following cessation of an exercise programme [12]. The degree of benefits might depend on complying with the prescribed intervention [13], yet some exercise intervention studies report attrition rates of up to 42% [14]. There is substantial evidence indicating that persons with MS are not engaging in sufficient amounts of physical activity for accruing health benefits; this questions the broad translation of exercise benefits amongst persons with MS. The loss of benefits after cessation of exercise interventions and the poor uptake of physical activity amongst persons with MS is a public health concern and might be associated with the “bedrock” of previous research.

We believe that one major limitation of previous research that ultimately undermines efficacy, effectiveness and translation is that the majority of studies on physical activity and its benefits have not undergone initial and systematic feasibility testing (i.e. of the processes, resources, management and scientific outcomes of clinical trials). Instead, previous pilot studies have been undertaken in controlled environments (e.g. laboratory settings) to primarily demonstrate efficacy, but may not translate to or achieve real life effectiveness for continued, long-term uptake of physical activity [15]. They may have poor compliance with the intervention itself. This lack of systematic feasibility testing may result in the main study not achieving efficacy or effectiveness for changing the target outcome. This means that researchers may not be conducting the preliminary work that informs the design and implementation of an intervention before examining if the intervention has an effect on an outcome of interest. Essentially, we are forgoing building a strong and stable foundation that supports a robust framework and scaffolding for our house; without such a foundation, we might have a house of cards!

There is growing acceptance and recognition of the importance of conducting and reporting preliminary work on the feasibility of a study. Feasibility studies might be considered similar to phase 1 or phase 2 evaluation (or “proof of concept”) of pharmacological interventions [16]. However, it is acknowledged that interventions involving behavioural or lifestyle interventions (e.g. physical activity) may not fit into the classical clinical trial developmental model [17] (i.e. phase 1, evaluation of intervention safety, safe dosage range and identify side effects; phase 2, evaluation of whether the intervention is effective; phase 3, confirmation of intervention effectiveness, monitoring of side effects, comparison with common alternative treatments and safety; phase 4, after market evaluation of the intervention’s effect in various populations and determination of side effects associated with long-term use [17]).

Feasibility is “an overarching concept for studies assessing whether a future study, project or development can be done” [18], and investigators attempt to answer a question about whether some element of the main study can and/or should be done, and, if so, how it could/should be done [16]. Pilot studies fit within the framework of feasibility [18], and investigators attempt to answer similar questions to a feasibility study and extend this to include a smaller scale version of the future study (i.e. a randomised or non-randomised trial). Feasibility studies of behavioural or lifestyle interventions (e.g. physical activity) fit within the stages of the updated National Institute of Health’s conceptual framework of intervention development [19], and these stages are outlined below and referred to in Fig. 1;

• Stage 0 includes the basic science of research and could include any behavioural, cognitive, affective or social or neuroscience being undertaken for development of the behavioural intervention.

• Stage I includes all activities related to the creation of a new intervention or the modification, adaption or refinement of an existing intervention. This stage includes feasibility and pilot testing.

• Stage II includes the testing of promising behavioural interventions in research settings, with research therapists/providers.

• Stage III includes the testing of intervention efficacy in a well-controlled, internally valid study in a community setting with community therapists/providers, and this includes community friendly fidelity monitoring and enhancement procedures as part of the intervention.

• Stage IV includes the testing of intervention effectiveness, and it examines behavioural intervention in community settings, with community therapists/providers, and maximises external validity.

• Stage V includes translation of the research. The translation involves implementing scientifically supported interventions into community settings and distributing intervention information and materials to the relevant groups [19].

Figure 1 contextualises the importance of feasibility studies within the larger process of developing and conducting clinical interventions. Stage I data is important to the advancement of stage II and stage III, while data from stage II to V can inform stage I for further refinement of the intervention [19]. For example, feasibility studies that target physical activity in a clinical population such as MS must consider metrics that are important for the success of future studies. These metrics provide information on if, and how, the research can be done in the first place. Further, stage I data will inform, and be informed by, the Basic Science data established in stage 0. Feasibility and pilot studies should be undertaken prior to the main stage II or stage III studies, [18], and we propose that information flow between these main stage II or stage III studies should inform alternate feasibility or pilot work; see Fig. 1. Such bidirectional informational flow will improve overall care for persons with MS and improve our understanding of clinical research in MS care.

Clinical journals are now accepting feasibility studies in recognition of the importance to the scientific field [20, 21]. Funding bodies recognise the importance of preliminary work and offer small grant programmes (e.g. the United States National Institute of Health Small Grant Program (RO3)). There are guidelines for conducting and reporting of preliminary studies [16, 22–24], and a framework has recently been proposed for defining feasibility and pilot studies [18].

Our objective is to investigate and clarify what feasibility processes have been investigated and described within the context of physical activity interventions in MS. We will undertake a systematic scoping review which will clarify the important definitions and conceptual boundaries of feasibility studies that provide context to the current position of preliminary research on this topic [25]. We provide examples of the process, resource, management and scientific metrics currently investigated in physical activity research in persons with MS and present the case that there are a number of important outcomes of feasibility that should be addressed in future research. We illustrate a case study of a pragmatic approach taken to assess the four feasibility metrics through a study of a physical activity and behavioural change intervention to increase the physical activity participation of persons with MS [26].

Our literature review was informed by past reviews of preliminary studies in clinical research [18, 20, 27, 28] and was carried out in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) [29]. Our inclusion criteria stipulated that papers had (1) to be published between 2000 and 2016, (2) to be written in English, (3) to study persons with MS, (4) to include interventions or assessments deemed to encapsulate physical activity and (5) to be indexed as a feasibility or pilot study. Additional exclusion criteria were (6) described as large scale (e.g. stage II or stage III studies), (7) discussion or review articles, (8) cross-sectional or retrospective data, (9) data gathering protocol only and (10) abstract only, and we further required that (11) the major consideration of the study was not to scientifically explore the treatment safety, dose, response and efficacy only (i.e. the effect of the intervention on the assessed outcomes). Our literature review was not designed to capture all preliminary studies in persons with MS that include aspects of physical activity; consistent with a scoping review [25, 30], it was designed to capture an overall summary of current research in this area that involves feasibility metrics.

We searched the following databases without date limits: PubMed, Science Direct, Scopus, SPORTDiscus and Web of Science. The key terms searched involved Multiple Sclerosis, acronyms of preliminary study (e.g. Safety, Feasibility, Pilot, Preliminary, Phase 1, Stage 1) and acronyms of physical rehabilitation (e.g. Physical activity, Exercise, Rehabilitation). An example of our search strategy is provided as an Appendix.

Two reviewers read each study and determined examples of process, resource, management or scientific feasibility and used those to evoke discussion on current trends in preliminary studies involving persons with MS and that included aspects of physical activity. To ensure agreement, we used a checklist to identify examples of process, resource, management and scientific feasibility from previous reviews and discussion papers [16, 23, 27, 31, 32]. We met frequently to discuss study content and make a decision on the representation of process, resource, management or scientific feasibility.

We initially planned to exclude all pilot studies that primarily reported upon the scientific outcome of treatment effect of studies, but, during our review, we established that other feasibility metrics were investigated in pilot studies, and we therefore chose to not exclude the studies.

Our literature search identified 205 studies (Fig. 2), and 82 were duplicates across search engines. We initially excluded 100 studies for the following reasons: described as large scale (i.e. stage II or stage II studies) (n = 4), discussion or review papers (n = 21), cross-sectional (n = 2) or retrospective (n = 2) studies; not inclusive of persons with MS (n = 4); did not included aspects of physical activity (n = 33); were data gathering protocols (n = 4); were abstracts only (n = 26); did not use the terms “feasibility” or “pilot” in the article title, abstract or keywords (n = 2) and were not written in English (n = 2). Two articles initially excluded due to not including the terms pilot or feasibility were reconsidered as these were secondary publications of a feasibility study.

There were 25 studies deemed of interest to inform the current status quo of preliminary physical activity studies in MS population. We read these studies and undertook a content analysis focused on our four feasibility metrics of interest, process, resource, management and scientific feasibility. Table 1 provides a summary of the four feasibility metrics, data which can be established during feasibility studies and examples of outcomes used to gather data. Table 1 further provides the detailed results of our feasibility case study.

Of the 25 articles, 21 included an intervention [7, 34–53], 1 compared recruitment methods into an intervention study [53] and 1 was a cost analysis of the same intervention study [54]. One study was an investigation into the feasibility of an outcome assessment [55], and one was a qualitative analysis of participant opinions following an intervention [56].

The words feasibility and pilot were used in the title of 6 [35, 37, 47, 48, 51, 52] and 8 [7, 34, 39–42, 44, 51] studies, respectively, and 23 [7, 35–54, 57] studies indexed the word feasibility in the abstract, and 7 [7, 35, 40–43, 48] studies included pilot in the abstract. One [35] study included feasibility in the key words, and one [52] study included pilot in the key words.

The reporting mechanism and a summary of results for each of the four feasibility domains are detailed in Table 1. We identified that aspects of process feasibility were included in 10 studies, aspects of resource feasibility were discussed in 21 studies, aspects of management feasibility were included in 4 studies and aspects of scientific outcome feasibility were included in 16 studies. Within our discussion of how past studies assessed the four feasibility domains, we further provided abbreviated results for these domains as assessed in our case study, Project GEMS [33].

Our discussion of the literature highlights that important areas of feasibility are not being adequately addressed in preliminary physical activity studies in persons with MS and this has major implications for the strength of the existing research regarding exercise and MS. We believe this might be addressed in future research through the provision of this scoping review of the literature. For example, process feasibility (e.g. recruitment) was discussed in less than half of the studies, and there is a need for more information on participant reactions to randomisation; it is not clear whether randomisation during clinical trials affects recruitment rates and how this might inform trial design (e.g. preference designs vs. RCTs). Metrics related to resources (e.g. time and resources) were explored in many studies, and this suggests that researchers are efficient at reporting these methodological areas. However, there is still a need for clarification of some aspects; for example, we do not know the costs involved in undertaking physical activity research programmes and these data are essential for both realistic application to funding bodies and the application of the intervention in everyday clinical practice. Data on management feasibility are not well reported in the included literature; without these data, replication of research studies and application of the intervention in clinical practice may be carried out in lieu of full knowledge regarding staffing and infrastructural requirements, and this may ultimately lead to failure of the intervention. Finally, there are many important aspects of scientific outcome feasibility, which require further explanation in preliminary studies, the reporting of intervention problems, and participant feedback will arm researchers with important knowledge to improve the science of physical activity interventions for persons with MS. There was a limitation of some studies to report hypothesis testing for baseline comparisons and the direct calculation of treatment effects, however these areas are not recommended in the new CONSORT extension guidelines [16].

Our case study, Projects GEMS, enabled us to systematically record feasibility study in the domains of process (e.g. recruitment), resource (e.g. communication with participants), management (e.g. missing data items) and scientific outcomes (e.g. compliance during the intervention and participant feedback). This approach allowed us to record challenges to the design, methods and procedures of the study. We designed the GEMS project to be consistent with guidelines for the conduction and reporting of preliminary studies [22–24]; however, we acknowledge that not all aspects of feasibility study were assessed during the GEMS project. Importantly, there are areas of feasibility such as determining ease of randomisation and providing details of variables which may threaten the validity of the research which are still to be discussed in the physical activity in persons with MS literature. We encourage researchers to monitor the four domains of feasibility when first designing behavioural interventions in persons with MS and to use these strong foundations to build improved interventions and better current research design.

The strength of the evidence indicating that exercise is beneficial for persons with MS is growing; however, the evidence is not yet unequivocal as there are some examples of exercise interventions being ineffective in improving the primary outcome in comparison with a control group, e.g. [7, 59], and/or exercise behaviour, potentially, being difficult to sustain [9–11]. It is of importance that we are not yet designing high impact studies using effective and sustainable interventions, and as a result, we are not articulating the exercise message clearly to the wider MS community. We believe that this can be addressed through feasibility research, and there is lack of a systematic approach for such trials in exercise and MS. Our review highlights the existing research and our recent study, and we hope that this serves as a clarion call for feasibility trials that will substantially strengthen the foundation or bed rock of research on exercise in MS. The time is ripe, and we invite your participation on feasibility research on exercise in MS.