Statistical Evidence in Psychological Networks: A Bayesian Analysis of 294 Networks from 126 Studies

Psychometric networks have become a popular tool for multivariate data analysis in psychology and the social sciences. Researchers conceptualize a construct as a network of variables, interpreting the presence or absence of a network edge (i.e., conditional independence) and the strength of the present edges (i.e., the strength of the partial associations). However, the statistical evidence supporting the network findings is generally not evaluated, and therefore it is unknown how robust the results in the network literature are. Bayesian methods allow us to answer this question by estimating the uncertainty about the network edges and the edge weights. Here, we estimate the uncertainty in the network field by analyzing 294 psychometric networks from 126 published papers with the Bayesian approach. We found inconclusive evidence for the presence or absence of one-third of the edges, weak evidence for half, and compelling evidence for less than twenty percent of the edges. Thus, 80% of edges from the analyzed networks lack sufficient support from data to conclude their presence or absence with confidence. Networks estimated on a high relative sample size, with more than 70 observations per possible edge, had sufficient evidence to conclude the presence or absence of more than half of its edges. Our study shows that networks are often supported by too little evidence from the data for results to be reported with confidence, not meaning that results are flawed but rather that they cannot provide a solid basis for cumulative science.All results are available in an accompanying open-access website ReBayesed allowing researchers to explore the reanalyzed networks and determine findings that are robust across studies.