Statistical Evidence in Psychological Networks

Psychometric network models have become increasingly popular in psychology and the social sciences as tools to explore multivariate data. In these models, constructs are represented as networks of observed variables, and researchers often interpret the presence or absence of edges as evidence for or against conditional associations between variables. However, the statistical evidence supporting these edges is rarely evaluated. Here we show that a large proportion of reported network findings are based on weak or inconclusive evidence. We reanalyzed 293 networks from 126 published papers using a Bayesian approach that quantifies the evidence for each edge. Across studies, one-third of edges showed inconclusive evidence (1/3 < BF_{10} < 3), about half showed weak evidence (BF_{10} > 3 or BF_{10} < 1/3), and fewer than twenty percent were strongly supported (BF_{10} > 10 or BF_{10} < 1/10). Networks based on relatively large sample sizes yielded more robust results. Our study shows that networks are often supported by too little evidence from the data for results to be reported with confidence, not meaning that results are flawed but rather suggest caution in interpreting individual edges.