Robust Bayesian meta-analysis: Model-averaging across complementary publication bias adjustment methods

Publication bias is a ubiquitous threat to the validity of meta-analysis and the accumulation of scientific evidence. In order to estimate and counteract the impact of publication bias, multiple methods have been developed; however, recent simulation studies have shown the methods performance to depend on the true data generating process, and no method consistently outperforms the others across a wide range of conditions. Unfortunately, when different methods lead to contradicting conclusions, researchers can choose those methods that lead to a desired outcome. To avoid the condition-dependent, allor-none choice between competing methods and conflicting results, we extend robust Bayesian meta-analysis and model-average across two prominent approaches of adjusting for publication bias: (1) selection models of p-values and (2) models adjusting for small-study effects. The resulting model ensemble weights the estimates and the evidence for the absence/presence of the effect from the competing approaches with the support they receive from the data. Applications, simulations, and comparisons to preregistered, multi-lab replications demonstrate the benefits of Bayesian model-averaging of complementary publication bias adjustment methods.

Automated summary

Publication bias is a significant threat to the validity of meta-analysis. Multiple methods have been developed to address this issue, but their performance depends on the data generating process. To overcome this problem, we propose a robust Bayesian meta-analysis and model-averaging approach that combines two prominent methods of adjusting for publication bias. This approach provides a more reliable estimation of the effect and evidence for its presence or absence.