Are semantic representations stable? A Bayesian framework

Researchers have begun using Bayesian hierarchical modeling to study semantic representations, for instance, in the context of natural language quantifiers such as most, few, and more than half. We propose a statistical model that disentangles three key semantic parameters: the meaning threshold of quantifiers, the vagueness surrounding meaning thresholds, and response noise. We use this model to test the stability of semantic representations over time and across different paradigms. To examine stability over time, we analyzed existing data (n=63) from Ramotowska et al. (2023). Contrary to the conclusions drawn by the original authors, we found overwhelming evidence in favor of the hypothesis that semantic representations change over time (BF > 10^304). At the same time, we found overwhelming evidence that the relative ordering of meaning thresholds within individuals remained stable (BF = 4 * 10^24). Next, we conducted a new experiment (n=178) to test stability across paradigms, specifically comparing a linguistic paradigm to a visual one. Here too, we found overwhelming support for differences in between-subject variability in meaning thresholds across paradigms (BF = 7.48 * 10^30) and for differences in vagueness (BF = 1.17 * 10^110). Our findings challenge the assumption that semantic representations of logical vocabulary have stable, fixed values, while suggesting that their relative ordering remains stable within individuals. The model we propose provides an effective framework for studying the semantics of quantifiers, detecting individual-level effects, and explicitly accounting for potential instability.