Modelling Uncertainty around Free-List Cultural Salience Scores

The free-list method and its companion metrics have enjoyed a remarkably productive history. Yet, most research using this method limits its value to informal comparisons that are heavily reliant upon point estimates such as item or cultural salience. In this paper we demonstrate a range of methods, including both bootstrapping and Bayesian approaches, to incorporate uncertainty into such group-level estimates. This approach involves: (i) resampling individual-level data (either based on replacement sampling via bootstrapping, or modelling the data distribution via intercept-only Bayesian regression models) to create a range of hypothetical alternative samples; (ii) generating the group-level estimate (e.g., Smith's $S$) in each sample; and (iii) using the variation in these estimates as uncertainty intervals. While we focus predominantly on cultural salience, the approach outlined here can be applied to other metrics as well. In addition to calculating uncertainty in such estimates, we also discuss and present some extensions to this approach, such as comparing estimates between items (e.g., whether the cultural salience of one trait differs from that of another) and between groups (e.g., whether cultural salience differs between males and females). We provide open data and code to help readers gain familiarity with these methods. Ultimately, we encourage researchers using free-list data to move beyond simply reporting point estimates.