Evaluating the Quality of Systematic Reviews and Meta-analyses Published in Behaviour Analysis Journals: An Umbrella Review

High-quality systematic reviews and meta-analyses are essential for translating evidence into practice. We conducted an umbrella review to evaluate the methodological quality of systematic reviews and meta-analyses published in the field of behaviour analysis up to and including 2023. Quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR-2) and the R(Revised)-AMSTAR instruments. Temporal trends were analysed using multilevel Bayesian regression. The protocol was preregistered on the Open Science Framework (doi.org/10.17605/OSF.IO/U38Z4). We identified 64 reviews (16 of which included a meta-analysis), all of which were rated as 'critically low' quality using the AMSTAR-2 criteria. Methodological shortcomings included absent protocol registration, inadequate risk of bias assessment, and failure to assess publication bias. Mean R-AMSTAR adherence was 43.8% (range 11-66%). Adherence increased by 1.08% per year across the study period (population-level average marginal effect from the mixed-effects logistic model (95% CI [0.59, 1.58]), indicating robust methodological improvement over time. Currently, many systematic reviews and meta-analyses in behaviour analysis do not yet meet contemporary standards of methodological rigour. Enhancing the quality, transparency and consistency of evidence syntheses is vital if systematic reviews are to meaningfully inform practice.