Are semantic representations stable? A Bayesian framework

Researchers have recently begun using Bayesian hierarchical modeling to study semantic representations, for instance, in the context of natural language quantifiers such as "most", "few", and "more than half". In this work, we propose a statistical model that disentangles three key semantic parameters: the meaning threshold of quantifiers, the vagueness surrounding meaning thresholds, and response noise. We use this statistical model to experimentally test the stability of semantic representations over time and across different paradigms. To examine stability over time, we first analyzed existing data (n=63) from Ramotowska et al. (2023). Contrary to the conclusions drawn by the original authors, we found overwhelming evidence in favor of the hypothesis that semantic representations change over time (BF > 10^304). At the same time, we found overwhelming evidence that the relative ordering of meaning thresholds within individuals remained stable (BF = 4 x 10^24). Next, we conducted a new experiment (n=178) to test stability across paradigms, specifically comparing a linguistic paradigm to a visual one. Here too, we found overwhelming support for differences in between-subject variability in meaning thresholds across paradigms (BF = 7.48 x 10^30) and for differences in vagueness (BF = 1.17 x 10^110). Our findings challenge the assumption that semantic representations are stable and open up a promising avenue for further research on the relative stability of semantic representations within individuals. Our results emphasize the need for models that can detect individual-level effects and explicitly account for potential instability. Our proposed model offers an effective framework for studying the semantic representation of quantifiers and provides the flexibility to be applied to a wide range of linguistic constructs.