Empirical comparison of subgroup effects in conventional and individual patient data meta-analyses

Objectives: Individual patient data (IPD) meta-analyses have been proposed as a major improvement in meta-analytic methods to study subgroup effects. Subgroup effects of conventional and IPD meta-analyses using identical data have not been compared. Our objective is to compare such subgroup effects using the data of six trials (n = 1,643) on the effectiveness of antibiotics in children with acute otitis media (AOM). Methods: Effects (relative risks, risk differences [RD], and their confidence intervals [Cl]) of antibiotics in subgroups of children with AOM resulting from (i) conventional meta-analysis using summary statistics derived from published data (CMA), (ii) two-stage approach to IPD meta-analysis using summary statistics derived from IPD (IPDMA-2), and (iii) one-stage approach to IPD meta-analysis where IPD is pooled into a single data set (IPDMA-1) were compared. Results: In the conventional meta-analysis, only two of the six studies were included, because only these reported on relevant subgroup effects. The conventional meta-analysis showed larger (age < 2 years) or smaller (age >= 2 years) subgroup effects and wider CIs than both IPD meta-analyses (age < 2 years: RD(CMA) -21 percent, RD(IPDMA-1) -16 percent, RD(IPDMA-2) -15 percent; age >= 2 years: RD(CMA) -5 percent, RD(IPDMA-1) -11 percent, RD(IPDMA-2) -11 percent). The most important reason for these discrepant results is that the two studies included in the conventional meta-analysis reported outcomes that were different both from each other and from the IPD meta-analyses. Conclusions: This empirical example shows that conventional meta-analyses do not allow proper subgroup analyses, whereas IPD meta-analyses produce more accurate subgroup effects. We also found no differences between the one- and two-stage meta-analytic approaches.