When you come to a fork in the road, take it: the Rashomon effect for social scientists

Social science researchers have developed many methods for determining which models to use to address social inquiries. While conventional approaches rely on either model selection (identifying a single "best" model) or model averaging (consulting weighted averages of multiple models), we argue that neither fully address the practical concerns of social science research. Instead, we propose addressing model uncertainty through the lens of the "Rashomon effect," which recognizes that many models may reflect the data equally well while making competing claims about the world. A set of nearly equally performing models on a given dataset is called a "Rashomon set." Given that models in the Rashomon set are essentially indistinguishable using statistical metrics, we turn to "post-fit criteria" to evaluate these similarly well-performing models, and articulate how researchers can use these criteria to navigate model multiplicity. We demonstrate the utility of this framework for tackling a classic research question, specifying the levels of a categorical independent variable. Specifically, we show how different aggregations of religious categories produce multiple plausible explanations for the association between religious affiliation and life satisfaction with data from the Faith Matters Survey. We contend that the Rashomon paradigm promotes more intellectually honest model selection while maintaining scientific rigor, arguing that model selection is an interpretive task that should be made more transparent.