- Editorial
- Open Access
- Published:
A website for pilot and feasibility studies: giving your research the best chance of success
Pilot and Feasibility Studies volume 5, Article number: 122 (2019)
Abstract
This editorial introduces a website for pilot and feasibility studies. Pilot and feasibility studies are about giving research the best chance of success, but must be performed well to have the greatest benefit. The website was developed by the Pilot and Feasibility Studies collaboration who developed the CONSORT Extension to Pilot and Feasibility Trials as a resource to help triallists and researchers perform well conducted pilot and feasibility studies. The website is also aimed at those interested in the latest methodology for these studies. We aim to keep the site updated with the latest publications and events related to pilot and feasibility studies and welcome feedback and suggestions from the research community on further resources or events to add.
Introduction
As an Associate Editor of Pilot and Feasibility Studies, it is clear from handling some of the submissions that there remains a need for improved conduct and reporting of pilot and feasibility studies. To address this need, and as a member of the Pilot and Feasibility Studies (PAFS) collaboration (see the ‘Acknowledgements’ section), who developed the CONSORT Extension to Pilot and Feasibility Trials [1, 2], we have designed a website to support those conducting pilot and feasibility studies and those carrying out methodological research on these types of studies. To our knowledge, there is no other similar online resource. The purpose of this editorial is to draw attention to the website. I briefly describe how the website was developed, introduce the content included in the website, and finally discuss plans for maintenance and development of the website as a resource for the research community and to which they can contribute.
What are pilot and feasibility studies?
There are an increasing number of studies referred to as pilot and feasibility studies. A feasibility study explores the question of whether something can be done, whether we should proceed with it, and if so, how. Pilot studies are a subset of feasibility studies and ask the same questions as feasibility studies but are additionally conducting part (or all) of a future study on a smaller scale. These definitions were the result of recent extensive research into the use of these terms [3].
Why perform pilot and feasibility studies?
The purpose of performing pilot and feasibility studies is to clarify any uncertainty about the feasibility of conducting a future study. Each study will have its own areas of uncertainties about feasibility. Some examples might include the feasibility of recruiting and retaining a sufficient number of patients, logistics of randomisation, testing the study protocol procedures, feasibility of data collection and which outcomes to use, and feasibility of implementing the intervention [4,5,6]. Findings from the pilot or feasibility study inform the design of the future study. Solutions can be found to problems encountered during the pilot or feasibility study, meaning that the research team are less likely to encounter problems in the definitive study and successful delivery is more likely. This is particularly important because main studies often require substantial time and resources, and there is increasing impetus to reduce research waste [7].
What is the current state of conduct and reporting?
Despite the importance of pilot and feasibility studies, and the progress made to date, their conduct and reporting still require improvement. For example, many authors conduct hypothesis tests for effectiveness, even though pilot and feasibility studies are not formally powered to assess potential effectiveness [5, 8]. Moreover, authors sometimes incorrectly use the effect size determined in a pilot study to power their main study [9, 10]. Ideally, authors should pre-specify progression criteria to indicate whether to proceed with the main trial, but many studies do not report any such criteria [11].
Reporting of pilot and feasibility studies requires improvement, as shown by several recent reviews [11,12,13]. For example, authors of pilot studies are not reporting their reason for the pilot well enough; reports should include a clear list of feasibility objectives, the rationale for sample size, and progression criteria for continuing to the main trial [11]. Some studies are currently incorrectly being labelled as ‘pilot’ or ‘feasibility’ studies when in fact they are small underpowered effectiveness studies [14]. In reporting of pilot and feasibility studies, it is important to show how the findings have informed the future study and to assist readers preparing for similar future studies. This is important whether the study leads to a full-scale trial or not and may be particularly important if the study failed so that other researchers can learn from what did and did not work and consider potential alternatives [15].
Why a website?
The idea for the website emerged as part of the on-going PAFS initiative [1, 2]. The call for improved conduct and reporting of pilot and feasibility studies has been addressed through several research papers, either highlighting the issue or providing advice and guidance [1,2,3,4,5,6]. However, we were not aware of anywhere where all the research on pilot and feasibility studies had been brought together and collated to provide an easily accessible resource for those conducting such studies. We first discussed the idea in March 2017 at one of the PAFS collaboration’s regular teleconferences and agreed the website would be funded through Sandra Eldridge’s NIHR Senior Investigator Award.
How was the website created?
The first outline of the structure of the website was created in April 2017, and members of the PAFS collaboration were allocated different sections to populate. Draft versions of the website were shared and updated via a series of emails and teleconferences. A web designer was employed in April 2018 to create the website based on the final draft of the contents.
In May 2018, an email invitation was sent out to stakeholders previously involved in the consensus meeting during the development of the CONSORT extension for pilot trials. Individuals were invited to review the website before it went live. Those who agreed were sent a link and password to the website at the end of June 2018 and a review sheet to complete. Feedback was received and the website updated. The site went live in September 2018.
What can I find on the website?
The website can be viewed at https://pilotandfeasibilitystudies.qmul.ac.uk/. The website comprises six pages: homepage, introduction page, resources page, noticeboard, who we are, and a contact page. The introduction page describes what pilot and feasibility studies are, including the distinction between internal and external pilot studies, provides examples of pilot and feasibility studies, reasons to perform such a study, and information on incorrect use of the terms pilot and feasibility. The resources page is divided into sections on design, analysis, and reporting, with further subsections within each of these. Resources are provided in the form of hyperlinks to published papers, with brief descriptive paragraphs as appropriate. The noticeboard page features news items, announcements, events, and courses concerned wholly or partly with pilot or feasibility studies, which might be of interest for people to attend, as well as details of past events and courses.
Who is the website for?
The website is aimed at anyone conducting a pilot or feasibility study (for example, triallists and health services researchers, clinicians, health professionals, grant funding bodies, peer reviewers, journal editors) as well as those carrying out methodological research (for example, researchers, statisticians, students). For those interested in conducting a pilot or feasibility study, there are helpful resources on how to choose the objectives for a study, advice on choosing the study design and sample size to match the chosen objectives, and resources to guide appropriate analysis and reporting of the results. For those wanting to carry out methodological research, there is information on the latest publications and guidelines related to pilot and feasibility studies, as well as information about other relevant events such as workshops and training courses.
How can I get involved?
We aim to keep the site updated with the latest publications and guidelines, as well as discussions and events related to pilot and feasibility studies. Minor updates are currently performed throughout the year, together with a formal annual review and update. We recognise that the website is unlikely ever to contain an exhaustive list of resources and encourage the research community to contribute additional material and relevant links using the ‘Contact us’ page on the website or sending an email to: pilotandfeasibilitystudies@qmul.ac.uk. We also welcome ideas for news items or events to add to our noticeboard.
Conclusion
Pilot and feasibility studies, if conducted well, are important for increasing the chance of successful delivery of the future study. However, current conduct of such studies is suboptimal. We have therefore developed a website providing resources to help researchers conduct a pilot or feasibility study. The website is designed to support those conducting pilot and feasibility studies using randomised and non-randomised designs. Moreover, those carrying out methodological research on these types of studies can also benefit from using the site. We aim to keep the site updated with the latest resources and events related to pilot and feasibility studies. We welcome feedback and suggestions from the research community on further resources that would be helpful to include, as well as items to add to our noticeboard.
Availability of data and materials
Not applicable.
References
- 1.
Eldridge S, Chan C, Campbell M, Bond C, Hopewell S, Thabane L, Lancaster GA. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud. 2016;2:64.
- 2.
Eldridge S, Chan C, Campbell M, Bond C, Hopewell S, Thabane L, Lancaster GA. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239.
- 3.
Eldridge S, Lancaster GA, Campbell M, Thabane L, Hopewell S, Coleman C, Bond C. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PloS One. 2016;11(3):e0150205.
- 4.
Arain M, et al. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010;10:67.
- 5.
Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.
- 6.
Thabane L, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1.
- 7.
Moher D, et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet. 2016;387(10027):1573–86.
- 8.
Sim J. Should treatment effects be estimated in pilot and feasibility studies? Pilot Feasibility Stud. 2019;5(1):107.
- 9.
Albers C, Lakens D. When power analyses based on pilot data are biased: inaccurate effect size estimators and follow-up bias. J Exp Soc Psychol. 2018;74:187–95.
- 10.
Kraemer HC, et al. Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry. 2006;63(5):484–9.
- 11.
Chan CL, Leyrat C, Eldridge SM. Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review. BMJ Open. 2017;7(11):e016970.
- 12.
Kannan S, Gowri S. Pilot studies: are they appropriately reported? Perspect Clin Res. 2015;6(4):207–10.
- 13.
Kristunas CA, et al. The current use of feasibility studies in the assessment of feasibility for stepped-wedge cluster randomised trials: a systematic review. BMC Med Res Methodol. 2019;19(1):12.
- 14.
Campbell MJ, Lancaster GA, Eldridge SM. A randomised controlled trial is not a pilot trial simply because it uses a surrogate endpoint. Pilot Feasibility Stud. 2018;4(1):130.
- 15.
Friedman L. Commentary: why we should report results from clinical trial pilot studies. Trials. 2013;14(1):14.
Acknowledgements
The author would like to acknowledge the support of the PAFS collaboration, both in the creation of the website and preparation of this editorial. The PAFS collaboration consists of Professor Sandra Eldridge and Claire Chan, Queen Mary University of London; Professor Mike Campbell, Sheffield University; Professor Lehana Thabane, McMaster University, Canada; Professor Christine Bond, Aberdeen University; Dr. Sally Hopewell, Oxford University; and Professor Gillian Lancaster, Keele University. The author would also like to acknowledge the helpful comments during website development from Daniel Shanahan (F1000), Professor Jenny Hewison (University of Leeds), and Dr. Chris Sutton (The University of Manchester). Finally, the author would like to acknowledge the web designer, Kevin Wade, and the blue cog image he found on istockphoto.com (image number: 640034264).
Funding
The website was funded through Sandra Eldridge’s NIHR Senior Investigator Award.
Author information
Affiliations
Contributions
CLC wrote this Editorial. The author read and approved the final manuscript.
Corresponding author
Correspondence to Claire Louise Chan.
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The author declares that she has no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Chan, C.L. A website for pilot and feasibility studies: giving your research the best chance of success. Pilot Feasibility Stud 5, 122 (2019). https://doi.org/10.1186/s40814-019-0522-6
Received:
Accepted:
Published:
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.