Table 1 Pilot research questions and implications for a full review

Determine the appropriate nomenclature for accurate identification of methodological reviews

Which search terms yield methodological reviews?

Identifying a list of terms that yields methodological reviews will inform the search strategy in the full review

Sensitivity/specificity ≥ 70%

Determine the need for methodological review reporting guidelines

Are research methods specified a priori?

Inconsistent pre-specification of methods would indicate the need for a full review

≤ 70% with published protocols

How many databases are searched?

Wide variation in the numbers of databases searched would indicate the need for a full review

Coefficient of variation ~ 1 (i.e., spread in results relative to the mean)

Are search time limits justified?

Inappropriate justification of search time limits would imply the need for a full review

≤ 70% justify search limits

Is the sample size justified?

Inappropriate justification of sample size for MRs designed as analytical studies (e.g., before-after comparisons, regression-based analyses) would imply the need for a full review

≤ 70% justify sample size or perform sample size calculation

Is a formal sample size calculation performed?

Inappropriate justification of sample size for MRs designed as analytical studies (e.g., before-after comparisons, regression-based analyses) would imply the need for a full review

Is a random sample of studies used?

Use of different sampling approaches to select a subset of studies from a larger group would indicate the need for a full review

Among studies where the goal was not to capture all available studies, ≤ 70% use a random sampling approach

Do research methods or authors suggest generalizable findings?

Lack of clear approaches to reporting generalizability would indicate the need for a full review

≤ 70% discuss the generalizability of findings