Effects of Publication Bias on the Moderator Effect in Meta-Analysis

Moderator analyses in meta-analysis are crucial for identifying study characteristics that predict the strength of associations and effects, such as contextual factors, that render interventions more or less effective. While unbiased estimation of moderator effects is required to draw valid conclusions for theory and practice and to steer future research efforts in the right direction, publication bias could invalidate moderator analysis. The effect of publication bias on the meta-analytic mean effect is well understood, but how it affects parameter estimates in fixed and mixed-effects meta-regression models is currently unclear. We assess analytically and using illustrative examples, how publication bias can distort moderator effect estimates and how this depends on the true effect sizes, the primary study sample sizes, and the publication bias severities at each moderator level. We demonstrate that a true moderator effect can be obscured and that a non-existent moderator effect can be induced when publication bias is present. Furthermore, we provide recommendations for identifying combinations of publication bias, primary study sample sizes, residual heterogeneity, and effect sizes that tend to hide existing or induce artificial moderator effects. Since the bias in moderator effects due to publication bias depends heavily on the characteristics of the meta-analysis, we also introduce a Shiny web application that applied researchers can use for sensitivity analyses.