A new measure of between-studies heterogeneity in meta-analysis

Assessing the magnitude of heterogeneity in a meta-analysis is important for determining the appropriateness of combining results. The most popular measure of heterogeneity, I-2, was derived under an assumption of homogeneity of the within-study variances, which is almost never true, and the alternative estimator, I, uses the harmonic mean to estimate the average of the within-study variances, which may also lead to bias. This paper thus presents a new measure for quantifying the extent to which the variance of the pooled random-effects estimator is due to between-studies variation, b, that overcomes the limitations of the previous approach. We show that this measure estimates the expected value of the proportion of total variance due to between-studies variation and we present its point and interval estimators. The performance of all three heterogeneity measures is evaluated in an extensive simulation study. A negative bias for b was observed when the number of studies was very small and became negligible as the number of studies increased, while I and I-2 showed a tendency to overestimate the impact of heterogeneity. The coverage of confidence intervals based upon b was good across different simulation scenarios but was substantially lower for I and I-2, especially for high values of heterogeneity and when a large number of studies were included in the meta-analysis. The proposed measure is implemented in a user-friendly function available for routine use in r and sas. b will be useful in quantifying the magnitude of heterogeneity in meta-analysis and should supplement the p-value for the test of heterogeneity obtained from the Q test. Copyright (c) 2016 John Wiley & Sons, Ltd.