Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review

Background: There is an increasing number of meta-analyses including data from non-randomized studies for therapeutic evaluation. We aimed to systematically assess the methods used in meta-analyses including non-randomized studies evaluating therapeutic interventions. Methods: For this methodological review, we searched MEDLINE via PubMed, from January 1, 2013 to December 31, 2013 for meta-analyses including at least one non-randomized study evaluating therapeutic interventions. Etiological assessments and meta-analyses with no comparison group were excluded. Two reviewers independently assessed the general characteristics and key methodological components of the systematic review process and meta-analysis methods. Results: One hundred eighty eight meta-analyses were selected: 119 included both randomized controlled trials (RCTs) and non-randomized studies of interventions (NRSI) and 69 only NRSI. Half of the meta-analyses (n = 92, 49 %) evaluated non-pharmacological interventions. Grey literature was searched for 72 meta-analyses (38 %). An assessment of methodological quality or risk of bias was reported in 135 meta-analyses (72 %) but this assessment considered the risk of confounding bias in only 33 meta-analyses (18 %). In 130 meta-analyses (69 %), the design of each NRSI was not clearly specified. In 131 (70 %), whether crude or adjusted estimates of treatment effect for NRSI were combined was unclear or not reported. Heterogeneity across studies was assessed in 182 meta-analyses (97 %) and further explored in 157 (84 %). Reporting bias was assessed in 127 (68 %). Conclusions: Some key methodological components of the systematic review process-search for grey literature, description of the type of NRSI included, assessment of risk of confounding bias and reporting of whether crude or adjusted estimates were combined-are not adequately carried out or reported in meta-analyses including NRSI.