Known unknowns and model selection in ecological evidence synthesis

Quantitative evidence synthesis is a prominent path towards generality in ecology. Generality is typically discussed in terms of central tendencies, such as an average effect across a compilation of studies, and the role of heterogeneity for advancing understanding is not as well developed. Heterogeneity quantifies the consistency of ecological effects across studies, and between-study heterogeneity is typically assumed as constant. Here, I relax the assumption of constant heterogeneity, and show how cross validation can further the generality goals of quantitative evidence synthesis. First, I examine scale-dependent heterogeneity for a meta-analysis of plant native-exotic species richness relationships, quantifying the relationships among unexplained effect size variation, spatial grain and extent. Second, I examine relationships among patch size, study-level covariates and unexplained variation in species richness using a database of fragmentation studies. Both case studies show that relaxing the assumption of constant unexplained variation can provide a more detailed description of available evidence - for example, where effects can be transferred with more or less certainty. Moreover, cross validation shows for both case studies that assuming constant heterogeneity can constrain model predictive performance, and in particular, the ability of model predictions to generalise to new studies.