Updating meta-analyses leads to larger type I errors than publication bias

Objective: To estimate the extent to which the practice of periodically updating meta-analyses causes inflation of the type I error and then to compare the estimate with the inflation caused by publication bias. We also present a simple method to adjust for the inflation associated with updating meta-analyses. Study Design and Setting: Simulations were used to estimate the error rates. Results: In general, updating meta-analyses caused 2- to 5-fold inflation of the type I error rates, which exceeded the inflation caused by publication bias. As a rule of thumb, the results of a meta-analysis are robust up to 5, 10, 15, or 22 updates, if the P-value multiplied by 4, 6, 8, or 10 remains below the desired significance level. Conclusion: Meta-analyses are likely to be updated until a clear conclusion is reached. Therefore, it is important to take the inflation of the error rate into account to interpret the results correctly. (C) 2009 Elsevier Inc. All rights reserved.