Apparently conclusive meta-analyses may be inconclusive-Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses

Background Random error may cause misleading evidence in meta-analyses. The required number of participants in a meta-analysis (i.e. information size) should be at least as large as an adequately powered single trial. Trial sequential analysis (TSA) may reduce risk of random errors due to repetitive testing of accumulating data by evaluating meta-analyses not reaching the information size with monitoring boundaries. This is analogous to sequential monitoring boundaries in a single trial. Methods We selected apparently conclusive (P <= 0.05) Cochrane neonatal meta-analyses. We applied heterogeneity-adjusted and unadjusted TSA on these meta-analyses by calculating the information size, the monitoring boundaries, and the cumulative Z-statistic after each trial. We identified the proportion of meta-analyses that did not reach the required information size and the proportion of these meta-analyses in which the Z-curve did not cross the monitoring boundaries. Results Of 54 apparently conclusive meta-analyses, 39 (72%) did not reach the heterogeneity-adjusted information size required to accept or reject an intervention effect of 25% relative risk reduction. Of these 39, 19 meta-analyses (49%) were considered inconclusive, because the cumulative Z-curve did not cross the monitoring boundaries. The median number of participants required to reach the required information size was 1591 (range, 339-6149). TSA without heterogeneity adjustment largely confirmed these results. Conclusions Many apparently conclusive Cochrane neonatal meta-analyses may become inconclusive when the statistical analyses take into account the risk of random error due to repetitive testing.