The power of the standard test for the presence of heterogeneity in meta-analysis

It has been suggested that the standard test for the presence of heterogeneity in meta-analysis has low power. Although this has been investigated using simulation, there is little direct analytical evidence of the validity of this claim. Using an established approximate distribution for the test statistic, a procedure for obtaining the power of the test is described. From this, a simple formula for the power is obtained. Although this applies to a special case, the formula gives an indication of the power of the test more generally. In particular, for a given significance level, the power can be calibrated in terms of the proportion of the studies' variances that is provided by between-study variation. A consideration of this quantity confirms that the test does, in general, have low power. It is suggested that practitioners, who wish to conduct the standard test, use the ideas provided in order to investigate the operating characteristics of the test prior to performing it. Copyright (c) 2005 John Wiley & Sons, Ltd.