Addressing Selective Reporting Bias in Meta-Analysis of Dependent Effect Sizes: A Tutorial in R

Selective reporting bias arises when findings from primary research studies are incompletely reported and where the likelihood that a result is reported depends on the magnitude or statistical significance of the effect. When applied to selectively reported data, conventional meta-analytic models can produce systematically biased parameter estimates. Various statistical methods have been proposed for dealing with selective reporting in univariate meta-analysis, where each primary study contributes a single effect size estimate. However, few methods have been developed for handling selective reporting and publication bias in meta-analysis with dependent effect sizes, which are a common feature of meta-analyses in psychology. In this tutorial, we provide a guide for how to investigate potential selective reporting in meta-analyses of dependent effects, focusing on diagnosis and correction of selection reporting bias. We review several recently developed methods including a regression-based adjustment technique, a step-function selection model, and a sensitivity analysis approach. We demonstrate the implementation of these methods in the R statistical environment using data from two recent meta-analyses as examples. We discuss the application and interpretation of these methods, highlighting their strengths and limitations.