Skip to main content
Download PDF

Design, analysis, and reporting of pilot studies in HIV: a systematic review and methodological study

Pilot and Feasibility Studies volume 7, Article number: 211 (2021) Cite this article

Abstract

Background

Pilot studies are essential in determining if a larger study is feasible. This is especially true when targeting populations that experience stigma and may be difficult to include in research, such as people with HIV. We sought to describe how pilot studies have been used to inform HIV clinical trials.

Methods

We conducted a methodological study of pilot studies of interventions in people living with HIV published until November 25, 2020, using Medline, Embase, and Cochrane Controlled Register of Trials (CENTRAL). We extracted data on their nomenclature, primary objective, use of progression criteria, sample size, use of qualitative methods, and other contextual information (region, income, level, type of intervention, study design).

Results

Our search retrieved 10,597 studies, of which 248 were eligible. The number of pilot studies increased steadily over time. We found that 179 studies (72.2%) used the terms “pilot” or “feasibility” in their title, 65.3% tested feasibility as a primary objective, only 2% used progression criteria, 23.9% provided a sample size estimation and only 30.2% used qualitative methods.

Conclusions

Pilot studies are increasingly being used to inform HIV research. However, the titles and objectives are not always consistent with piloting. The design and reporting of pilot studies in HIV could be improved.

Peer Review reports

Introduction

Wastefulness in medical research is a major concern for researchers and funders and has been estimated to be at 85% of research investment [1]. There are several contributors to this waste. These include researchers not asking relevant questions, study results being inaccurately reported, and the inappropriate use of study design [2]. Recent work has demonstrated that pilot studies are very effective in reducing waste [3]. It was found that by employing pilot studies, the UK’s National Institute for Health Research’s (NIHR) Research for Patient Benefit (RfPB) Program saved approximately £20m, as otherwise non-feasible studies would have been conducted [3]. Pilot studies are especially useful in fields where participants are difficult to recruit and retain. However, despite the recognized value of pilot studies, there is still considerable confusion surrounding what constitutes a pilot study, how they should be designed, and how researchers decide whether they should proceed with the full study [4].

A pilot study is often described as a scaled down version of a larger study with feasibility, the assessment of the ability to conduct the full-scale study, as the primary goal [5]. Given the numerous challenges of recruiting and retaining participants in HIV research, pilot studies are particularly important in this field [6,7,8,9,10,11,12,13,14,15]. In this context, pilot studies could help researchers understand how they can adjust their procedures and reduce waste, especially when working with populations that experience difficulties, including stigma and discrimination [3]. People with HIV may belong to other key populations and face additional social stigma (e.g., men who have sex with men [MSM], commercial sex workers [CSWs], or people who inject drugs [PWID]), making participation in research more difficult [8,9,10,11,12, 16]. Studies have found that certain subpopulations (e.g., women, PWID, and African, Caribbean and Black (ACB) peoples) have dropout rates in research studies ranging from 30 to 50% [17,18,19].

The primary goal of this study was to describe the design, analyses, and reporting of pilot studies in HIV. The main outcomes of interest were the following:

  1. 1.

    The nomenclature of pilot studies (“pilot” or “feasibility” in the title)

  2. 2.

    Their declared primary objective (feasibility or effectiveness) and primary outcome

  3. 3.

    The use of progression criteria (criteria that would inform the decision to move to a larger trial)

  4. 4.

    Sample size estimation or justification (a description of why the sample size was chosen)

  5. 5.

    Use of qualitative methods (inclusion of qualitative assessments to inform feasibility)

  6. 6.

    The inclusion of key populations as defined by United Nations Program on HIV/AIDS (UNAIDS) [20,21,22,23]

Outcomes 1-5 have been identified as common shortcomings in pilot studies [6,7,8,9,10,11,12,13,14,15, 24,25,26,27,28,29,30,31,32,33,34,35]. The inclusion of key populations would indicate that a more diverse sample of people with HIV was included.

Methods

We conducted a methodological study of pilot studies in the HIV literature as per the guidelines reported by Murad and Wang for reporting meta-epidemiological research [36].

Ethics

This study used publicly available secondary data and therefore ethics review was not required.

Criteria for inclusion

All pilot studies of interventions conducted exclusively in people with HIV and published in English were eligible. We included randomized and non-randomized studies, using mixed or quantitative methods with at least one feasibility outcome [4]. Outcomes were deemed to be feasibility outcomes if they fit into a category outlined by Thabane et al. [4]. These categories include (1) assessing the processes involved in the study, (2) evaluation of resources required for the study, (3) management of potential human and data management problems, and (4) assessment of intervention safety, dose response, and variance of effect.

Search method for identifying pilot studies

We conducted an exhaustive search of the following databases: Medline, Embase, and the Cochrane Central Register of Controlled Trials (CENTRAL). These databases were searched from inception to November 25, 2020. Our search strategy was developed in collaboration with a librarian at the library services of the McMaster Health Sciences Central Library. The key concepts included in the search were “pilot,” “feasibility,” “proof-of-concept,” “exploratory,” “preliminary,” and “HIV.” The complete search strategies are included in the supplementary materials.

Screening and data extraction

We compiled the references and removed duplicate citations using Endnote X9 reference manager software [37]. We screened the remaining references first by their title and abstract and then by examining their electronic full texts. Both screening steps were done in duplicate by two independent reviewers using the Covidence data management platform for systematic reviews provided by McMaster University [38]. The reviewers attempted to resolve discrepancies by discussion and included a third reviewer if consensus could not be reached.

Data from included references were extracted using a piloted data-extraction form on RedCap [39]. Basic bibliometric information extracted from the studies included the following: the first author’s last name, study title, year of publication, journal of publication, and country of study (both region and income level). Region was determined using the regional groupings definitions provided by the World Health Organization (WHO), and income level was determined as per the World Bank Criteria [40, 41]. Other information collected included presence of a feasibility/pilot-identifying term in the title, study objectives, whether feasibility was a primary outcome, the use of progression criteria, a sample size justification, qualitative methods and study design, intervention type, and the inclusion of key populations (as defined by the UNAIDS and WHO [20,21,22,23]); key populations for which data were collected included (1) PWID, (2) MSM, (3) incarcerated populations, (4) CSW, (5) pregnant women, (6) children, (7) youth, (8) indigenous people, (9) ACB people, (10) women, and (11) transgender people. Progression criteria, having a feasibility related primary outcome and key labeling of pilot study status, are key characteristics of a pilot study. This information was extracted as these study characteristics have been found to impact study outcomes [42, 43]. We contacted authors via email to clear up ambiguity or to collect missing data.

Analysis

We conducted a descriptive analysis and reported counts and percentages for categorical variables and median (minimum, maximum) for continuous variables.

Results

Results of search

Our search returned 10,597 articles for title and abstract review. Of these, 536 were retrieved for full-text review. Only 248 articles met our eligibility criteria. Figure 1 is a flow chart of our screening and selection procedures.

Fig. 1
figure 1

Flow chart of study screening and selection

Full size image

Characteristics of included studies

The 248 studies included in our review were published between 1998 and 2020, with a steady increase over time (Fig. 2). Less than half of the included studies were randomized (108/43.5%). The majority investigated a pharmaceutical intervention (227/91.5%); were conducted in the Americas (137/55.2%) or Africa (71/28.6%). Further characteristics are reported in Table 1.

Fig. 2
figure 2

Yearly trends in publication of pilot studies in the HIV literature

Full size image
Table 1 Baseline characteristics of included studies
Full size table

Outcomes

Nomenclature

This has been clarified. New sentence: “Most studies were easily identifiable as pilot or feasibility studies, with 179 studies (72.2%) including the terms pilot, feasibility, or a feasibility outcome in the study title to denote the pilot status.” Feasibility outcomes were used to denote pilot status in 27 of these studies (15.1%). This was often done with the feasibility outcome “acceptance.” For example, a study was titled “Acceptability of a trial of vaginal progesterone for the prevention of preterm birth among HIV-infected women in Lusaka, Zambia: A mixed methods study” [44]. The remaining 69 studies (27.8%) had no indication of their pilot nature in the title.

Objectives

Study feasibility objectives were often clearly stated in the beginning or at the conclusion of the introduction. The primary objective was feasibility in 162 studies (65.3%). The remaining studies had goals centered around informing the sample size of the larger study, assessing efficacy, intervention development and to assess the reliability of a measure, with feasibility treated as a secondary outcome.

Outcomes

Feasibility outcomes were used as a primary outcome in 157 (63.3%) studies. We found that the feasibility outcomes assessed in the studies in this review could be grouped into 11 categories in total (Table 2). The most common outcomes were acceptance and retention of participants (180/72.6%; 135/54.4%, respectively), as well as evaluating participant enrolment (106/74.6%) and compliance (131/52.8%) to the intervention and study procedures. Trialists also often sought feedback (90/36.3%), primarily from participants and occasionally from study staff. Miscellaneous feasibility outcomes were also assessed and defined in 41 studies. The most common of these were implementation (34/13.7%), intervention initiation (6/2.4%), and consent rate (1/0.4%). The same outcome was defined differently in most instances, see Table 2. The CONSORT extension for pilot studies distinguishes between primary and secondary feasibility outcomes. The former are outcomes that inform the decision about progressing to a full RCT. Secondary feasibility outcomes are those related to patient centered outcomes [45].

Table 2 Feasibility outcomes and definitions
Full size table

Samples size estimation/justification

A sample size estimation was provided in 59 studies (23.8%). Forty-two of these studies (71.1%) provided an appropriate justification. Sample size justifications could be grouped into 6 categories. The most common justification was the use of a conventional sample size calculation with the intervention effect size to calculate a suitable sample size (20/33.9%). Researchers also equally relied on similar studies (6/10.2%) and on recommendations in the literature (6/10.2%). Three (5.1%) studies determined their sample size based on the resources available to conduct the study. Two (3.4%) studies used a proportion of the sample size of the larger study to justify their estimation. Finally, miscellaneous justifications were provided in 5 (10.2%) studies. For example, Tsima et al. justified their sample size using their estimation of expected recruitment rate [46].

Progression criteria

Only 5 (2%) of the included studies mentioned the progression criteria for their pilot studies. These studies were all RCTs published in 2020. Of these 5 studies, 3 (60%) were conducted in African countries. Of the 3 conducted in African countries, 2 (40%) were in low-middle-income countries and the other (1/20%) conducted in an upper-middle-income country. The other two were conducted in Europe and America.

Qualitative methods

Qualitative methods were included in only 75 studies (30.2%). Analysis was primarily conducted using data collected from participants. However, some studies also collected data from staff as well (25/33.33%).

Key populations

Close to half of the studies included a key population (134/54.0%). The complete composition of the patient sample is found in Table 1. ACBs represented the largest subpopulation among studies including key populations at 57% (n = 77). Only 24 (17.9%) of studies sought to purposely recruit women. Youth (28/17.2%) and MSM (20/14.9%) made up the remaining prominent vulnerable populations present. Other vulnerable populations present were PWID, CSW, pregnant women, children, and transgender women.

Discussion

To the best of our knowledge, this is the first methodological study of pilot studies in the HIV literature. We found that although pilot studies are becoming increasingly common in the HIV literature, there are considerable gaps in how they are labeled, designed, and how their findings are reported.

It is important to clearly label pilot studies and to use the correct terminology when doing so. Making pilot studies easier to identify helps inform readers that the primary goal of the study is to assess feasibility. We found that most authors (179/72.2%) labeled their studies clearly in the title.

Other reviews have found similar results. In two reviews of pilot studies in the cluster RCT literature and of pilot studies in the Clinical Rehabilitation journal, 83% and 87% of studies contained the terms pilot or feasibility in their title, respectively [24, 25]. The latter found that more than half of the studies used the pilot and feasibility terms interchangeably [25].

Our findings regarding authors replacing the primary feasibility objectives of their pilot studies, with other outcome(s) unrelated to feasibility, are in line with those of other authors [25,26,27]. The primary objective of a pilot or feasibility study must be to assess the feasibility of a larger study. As a result, assessing efficacy in a pilot study is inappropriate as this is not the primary goal of pilot studies and they are not powered to do so. However, smaller, non-pilot studies are important as they are important in hypothesis generation and in challenging widely held beliefs and common practices [47]. These studies should be encouraged and labeled properly as they also have a place in the literature [47]. The CONSORT extension for pilot studies of RCTs reinforces the requirement that pilot RCTs must have feasibility as their primary outcome by requiring feasibility outcomes be pre-specified and clearly defined [45].

Pilot studies must outline how authors intend to use their findings to inform future steps with pre-specified progression criteria [4]. However, only 5 (2%) studies assessed in this review included such criteria. Not reporting progression criteria is problematic, as we are unable to evaluate the criteria which the authors used to base their decision. A recent methodological review found that only 19.8% of studies included progression criteria [28]. Meanwhile, a review of cluster RCT pilot studies found that 89% of studies specified progression criteria [27]. However, the latter found that only 17% justified the criteria [27].

Currently, the CONSORT extension for pilot studies of RCTs requires reporting progression criteria, if applicable [45]. However, progression criteria are a key requirement of pilot studies, and this criterion should be strengthened to require the specification of the prespecified criteria. Progression criteria would improve the interpretability of the study [45]. The pilot studies assessed lacked both estimations and justifications for their sample sizes. Only 23.8% (n = 59) of studies had estimations for their sample size. However, even among these studies, 28.8% (n = 17) did not justify them. Chan et al. obtained similar results as they found that only 44% of pilot cluster RCTs justified for their sample size (27). As pilot studies do not aim to test hypotheses, formal power considerations are not necessary. However, it is still necessary to justify the sample size selected. As for now, guidance is only available for RCTs. As a result, concerns relating to progression criteria pertain more to pilot trials and are not always relevant for all pilot studies.

Several approaches can be used, such as targeting a percentage of the larger study’s sample size [48], having a set minimum number of participants per arm [48, 49], and using a stepped approach determined by standardized effect sizes [50]. Viechtbauer et al. have also proposed a sample size equation using the probability of a specific problem occurring during the trial to determine a sample size [51].

There have been recent calls for greater inclusion of qualitative methods in pilot studies [52, 53]. The use of qualitative methods has been shown to help refine study procedures, including optimizing recruitment and retention [29, 30]. By incorporating these methods in pilot studies, investigators are able to set realistic targets, craft pragmatic procedures, and ask and answer a wider range of questions, while gaining granular detail [53, 54]. As mentioned above, identification of pilot studies may be challenging given sub-optimal labeling. More so, identification of feasibility and pilot studies containing qualitative methods may be harder to identify as the qualitative components may be reported separately. These qualitative papers are harder to find if their titles do not contain the terms “pilot” or “feasibility.”

The HIV patient population is diverse and people with HIV, a stigmatized population, often belong to other stigmatized groups as well. This includes MSM, ACB, PWIDs, and CSWs, and is harder to recruit and retain in studies [6,7,8,9,10,11,12,13,14,15]. This impacts the ability to study these populations [6,7,8,9,10,11,12,13,14,15,16,17,18]. As a result, it is important to employ strategies that reduce barriers to participation and to evaluate them using pilot studies before committing to a full study [33, 55]. Few studies included these key populations, and this could potentially compromise feasibility in the larger study as the challenges in recruiting and retaining them have not been investigated in the pilot.

Our meta-epidemiological study has some weaknesses. Firstly, we were reliant on the authors’ conclusions to determine study feasibility and progression to the larger study. In addition, some of the studies included were evaluated using methodological advances developed after they were published, and this may explain why these approaches (sample size estimation, progression criteria etc.) were not used.

While our study does have weaknesses, it also has several strengths. Our review was robust as our search was highly sensitive, as demonstrated by the exceptionally large number of studies screened (n = 9297). The concepts searched, pilot studies and HIV, were purposefully broad as we are interested in all HIV interventions. In addition, with our study being specific to the HIV literature, we are able to evaluate a particular area of research where pilot studies will increasingly play a bigger role.

Future studies could expand on this work and evaluate the impact pilot studies have on the final study. Furthermore, future studies can assess the change in study quality since the introduction of the CONSORT extension for pilot studies.

Conclusion

Pilot studies are increasingly being used in the HIV field. However, feasibility outcomes were not always the primary outcomes of the pilot studies evaluated. In addition, many key pilot study requirements, such as selecting a sample size, crafting progression criteria, and defining feasibility outcomes were often not included. Higher quality pilot studies are needed.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

NIHR:

National Institute for Health Research

RfPB:

Research for patient benefit

HIV:

Human immunodeficiency virus

CENTRAL:

Cochrane Central Register of Controlled Trials

WHO:

World Health Organization

RCT:

Randomized controlled trial

ART:

Antiretroviral therapy

ACB:

African, Caribbean, and Black

MSM:

Men who have sex with men

PWID:

People who inject drugs

CSW:

Commercial sex workers

STROBE:

STrengthening the Reporting of Observational studies in Epidemiology

NGT:

Nominal group technique

References

  1. Glasziou P. The role of open access in reducing waste in medical research. PLoS Med. 2014;11(5):e1001651.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Obstet Gynecol. 2009;114(6):1341–5.

    Article  PubMed  Google Scholar 

  3. Morgan B, Hejdenberg J, Hinrichs-Krapels S, Armstrong D. Do feasibility studies contribute to, or avoid, waste in research? PLoS One. 2018;13(4):e0195951.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One. 2016;11(3):e0150205.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Yehia BR, Stewart L, Momplaisir F, Mody A, Holtzman CW, Jacobs LM, et al. Barriers and facilitators to patient retention in HIV care. BMC Infect Dis. 2015;15:246.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Hoffman KA, Baker R, Kunkel LE, Waddell EN, Lum PJ, McCarty D, et al. Barriers and facilitators to recruitment and enrollment of HIV-infected individuals with opioid use disorder in a clinical trial. BMC Health Serv Res. 2019;19(1):862.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Anastasi JK, Capili B, Kim GH, Chung A. Clinical trial recruitment and retention of a vulnerable population: HIV patients with chronic diarrhea. Gastroenterol Nurs. 2005;28(6):463–8.

    Article  PubMed  Google Scholar 

  9. De La Rosa M, Babino R, Rosario A, Martinez NV, Aijaz L. Challenges and strategies in recruiting, interviewing, and retaining recent Latino immigrants in substance abuse and HIV epidemiologic studies. Am J Addict. 2012;21(1):11–22.

    Article  Google Scholar 

  10. El-Sadr W, Capps L. The challenge of minority recruitment in clinical trials for AIDS. JAMA. 1992;267(7):954–7.

    Article  CAS  PubMed  Google Scholar 

  11. Fortune T, Wright E, Juzang I, Bull S. Recruitment, enrollment and retention of young black men for HIV prevention research: experiences from the 411 for safe text project. Contemp Clin Trials. 2010;31(2):151–6.

    Article  PubMed  Google Scholar 

  12. Silvestre AJ, Hylton JB, Johnson LM, Houston C, Witt M, Jacobson L, et al. Recruiting minority men who have sex with men for HIV research: results from a 4-city campaign. Am J Public Health. 2006;96(6):1020–7.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Cook C, Mack J, Cottler LB. Research participation, trust, and fair compensation among people living with and without HIV in Florida. AIDS Care. 2018;30(1):27–31.

    Article  PubMed  Google Scholar 

  14. Menezes P, Eron JJ Jr, Leone PA, Adimora AA, Wohl DA, Miller WC. Recruitment of HIV/AIDS treatment-naive patients to clinical trials in the highly active antiretroviral therapy era: influence of gender, sexual orientation and race. HIV Med. 2011;12(3):183–91.

    Article  CAS  PubMed  Google Scholar 

  15. Nalubega S, Evans C. Participant views and experiences of participating in HIV research in sub-Saharan Africa: a qualitative systematic review. JBI Database System Rev Implement Rep. 2015;13(5):330–420.

    Article  PubMed  Google Scholar 

  16. Gemmill R, Williams AC, Cooke L, Grant M. Challenges and strategies for recruitment and retention of vulnerable research participants: promoting the benefits of participation. Appl Nurs Res. 2012;25(2):101–7.

    Article  PubMed  Google Scholar 

  17. Batista P, Deren S, Banfield A, Silva E, Cruz M, Garnes P, et al. Challenges in recruiting people who use drugs for HIV-related biomedical research: perspectives from the field. AIDS Patient Care STDs. 2016;30(8):379–84.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Loutfy MR, Logan Kennedy V, Mohammed S, Wu W, Muchenje M, Masinde K, et al. Recruitment of HIV-positive women in research: discussing barriers, facilitators, and research personnel’s knowledge. Open AIDS J. 2014;8:58–65.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Asiimwe SB, Kanyesigye M, Bwana B, Okello S, Muyindike W. Predictors of dropout from care among HIV-infected patients initiating antiretroviral therapy at a public sector HIV treatment clinic in sub-Saharan Africa. BMC Infect Dis. 2016;16:43.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Bekker LG, Johnson L, Wallace M, Hosek S. Building our youth for the future. J Int AIDS Soc. 2015;18(2 Suppl 1):20027.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Macdonald V, Verster A, Baggaley R. A call for differentiated approaches to delivering HIV services to key populations. J Int AIDS Soc. 2017;20(Suppl 4):21658.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Djomand G, Quaye S, Sullivan PS. HIV epidemic among key populations in West Africa. Curr Opin HIV AIDS. 2014;9(5):506–13.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Rao A, Stahlman S, Hargreaves J, Weir S, Edwards J, Rice B, et al. Sampling key populations for HIV surveillance: results from eight cross-sectional studies using respondent-driven sampling and venue-based snowball sampling. JMIR Public Health Surveill. 2017;3(4):e72.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Castrillo-Viguera C, Grasso DL, Simpson E, Shefner J, Cudkowicz ME. Clinical significance in the change of decline in ALSFRS-R. Amyotroph Lateral Scler. 2010;11(1-2):178–80.

    Article  PubMed  Google Scholar 

  25. Kaur N, Figueiredo S, Bouchard V, Moriello C, Mayo N. Where have all the pilot studies gone? A follow-up on 30 years of pilot studies in clinical rehabilitation. Clin Rehabil. 2017;31(9):1238–48.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Shanyinde M, Pickering RM, Weatherall M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Med Res Methodol. 2011;11:117.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Chan CL, Leyrat C, Eldridge SM. Quality of reporting of pilot and feasibility cluster randomised trials: a systematic review. BMJ Open. 2017;7(11):e016970.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Mbuagbaw L, Kosa SD, Lawson DO, Stalteri R, Olaiya OR, Alotaibi A, et al. The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study. Pilot Feasib Stud. 2019;5:120.

    Article  Google Scholar 

  29. Donovan JL, Rooshenas L, Jepson M, Elliott D, Wade J, Avery K, et al. Optimising recruitment and informed consent in randomised controlled trials: the development and implementation of the quintet recruitment intervention (QRI). Trials. 2016;17(1):283.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Elliott D, Husbands S, Hamdy FC, Holmberg L, Donovan JL. Understanding and improving recruitment to randomised controlled trials: qualitative research approaches. Eur Urol. 2017;72(5):789–98.

    Article  PubMed  Google Scholar 

  31. UyBico SJ, Pavel S, Gross CP. Recruiting vulnerable populations into research: a systematic review of recruitment interventions. J Gen Intern Med. 2007;22(6):852–63.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Gama A, Martins MO, Dias S. HIV research with men who have sex with men (MSM): advantages and challenges of different methods for most appropriately targeting a key population. AIMS Public Health. 2017;4(3):221–39.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Castillo-Mancilla JR, Cohn SE, Krishnan S, Cespedes M, Floris-Moore M, Schulte G, et al. Minorities remain underrepresented in HIV/AIDS research despite access to clinical trials. HIV Clin Trials. 2014;15(1):14–26.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Schwartz SR, Kavanagh MM, Sugarman J, Solomon SS, Njindam IM, Rebe K, et al. HIV viral load monitoring among key populations in low- and middle-income countries: challenges and opportunities. J Int AIDS Soc. 2017;20 Suppl 7:43–49.

  35. Jenkins RA. Recruiting substance-using men who have sex with men into HIV prevention research: current status and future directions. AIDS Behav. 2012;16(6):1411–9.

    Article  PubMed  Google Scholar 

  36. Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139–42.

    Article  PubMed  PubMed Central  Google Scholar 

  37. The EndNote Team. EndNote. EndNote X7 ed. Philadelphia: Clarivate Analytics; 2013.

    Google Scholar 

  38. Innovation VH. Covidence systematic review software. Melbourne: COVIDENCE; 2014.

  39. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  PubMed  Google Scholar 

  40. Organization WH. Definition of regional groupings 2020. Geneva: WHO; 2020.

  41. Bank TW. World Bank open data. World Bank. 2018.

  42. Faber J, Fonseca LM. How sample size influences research outcomes. Dental Press J Orthod. 2014;19(4):27–9.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Baldeh T, MacDonald T, Kosa SD, Lawson DO, Stalteri R, Olaiya OR, et al. More pilot trials could plan to use qualitative data: a meta-epidemiological study. Pilot Feasib Stud. 2020;6(1):164.

    Article  Google Scholar 

  44. Price JT, Mabula-Bwalya CM, Freeman BL, Carda-Auten J, Phiri WM, Chibwe K, et al. Acceptability of a trial of vaginal progesterone for the prevention of preterm birth among HIV-infected women in Lusaka, Zambia: a mixed methods study. PLoS One. 2020;15(9):e0238748.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasib Stud. 2016;2:64.

    Article  Google Scholar 

  46. Tsima BM, Moedi P, Maunge J, Machangane K, Kgogwane M, Mudojwa T, et al. Feasibility of implementing a novel behavioural smoking cessation intervention amongst human immunodeficiency virus-infected smokers in a resource-limited setting: a single-arm pilot trial. South Afr J HIV Med. 2020;21(1):1075.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Sackett DL, Cook DJ. Can we learn anything from small trials? Ann N Y Acad Sci. 1993;703:25–31 discussion -2.

    Article  CAS  PubMed  Google Scholar 

  48. Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013;66(2):197–201.

    Article  PubMed  Google Scholar 

  49. Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15:264.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016;25(3):1057–73.

    Article  PubMed  Google Scholar 

  51. Viechtbauer W, Smits L, Kotz D, Bude L, Spigt M, Serroyen J, et al. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015;68(11):1375–9.

    Article  PubMed  Google Scholar 

  52. Hallingberg B, Turley R, Segrott J, Wight D, Craig P, Moore L, et al. Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance. Pilot Feasib Stud. 2018;4:104.

    Article  Google Scholar 

  53. Bertram W, Moore A, Wylde V, Gooberman-Hill R. Optimising recruitment into trials using an internal pilot. Trials. 2019;20(1):207.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. O’Cathain A, Hoddinott P, Lewin S, Thomas KJ, Young B, Adamson J, et al. Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot Feasib Stud. 2015;1:32.

    Article  Google Scholar 

  55. Wolf RC, Bingham T, Millett G, Wilcher R. Building the evidence base to optimize the impact of key population programming across the HIV cascade. J Int AIDS Soc. 2018;21 Suppl 5:e25146.

    PubMed  Google Scholar 

Download references

Acknowledgements

None

Funding

No funding was used for this study.

Author information

Authors and Affiliations

  1. Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4L8, Canada

    Hussein Ali El-Khechen, Shakib Akhter & Lawrence Mbuagbaw

  2. Biostatistics Unit, Father Sean O’Sullivan Research Centre, St Joseph’s Healthcare, Hamilton, Ontario, Canada

    Mohammed Inam Ullah Khan & Lawrence Mbuagbaw

  3. Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada

    Selvin Leenus & Alvin Leenus

  4. Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada

    Oluwatobi Olaiya

  5. Joan C. Edwards School of Medicine, Marshall University, Huntington, WV, USA

    Zoha Durrani

  6. Department of Kinesiology, Faculty of Science, McMaster University, Hamilton, Ontario, Canada

    Zaryan Masood

  7. Center for the Development of Best Practices in Health, Yaoundé, Cameroon

    Lawrence Mbuagbaw

Authors
  1. Hussein Ali El-Khechen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Inam Ullah Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Selvin Leenus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Oluwatobi Olaiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zoha Durrani
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zaryan Masood
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alvin Leenus
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shakib Akhter
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Lawrence Mbuagbaw
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HE and LM designed the study. HE wrote the first draft. HE and LM analyzed the data. HE, MK, SL, OO, ZD, ZM, and AL participated in the title and abstract screening. HE, ZD, OO, and SL participated in the full-text screening. HE and SA extracted the data. HE and LM revised several versions of the manuscript. All authors reviewed the final version of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hussein Ali El-Khechen.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:.

Search Strategies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

El-Khechen, H.A., Khan, M.I.U., Leenus, S. et al. Design, analysis, and reporting of pilot studies in HIV: a systematic review and methodological study. Pilot Feasibility Stud 7, 211 (2021). https://doi.org/10.1186/s40814-021-00934-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40814-021-00934-9

Keywords

Pilot and Feasibility Studies

ISSN: 2055-5784