Expertise Shapes the Multidimensional Perception of Stories

Stories are a powerful form of language that connect people, evoke memories, and spark imagination. When reading a story, people can appreciate it on a range of dimensions, such as how creative, complex, and coherent it is. However, many previous studies have focused on the evaluation of story creativity specifically, relying on experts’ ratings and high inter-rater reliability, which obscures meaningful differences between ratings across populations and misses the opportunity to assess other dimensions of narrative quality (e.g., cohesiveness). Moreover, although a few quantitative linguistic features have recently been used to predict creativity ratings of stories, it is unclear how those fine-grained features drive a broader range of narrative qualities, as well as individual differences in human ratings. Here, we addressed these limitations by examining the impact of expertise on ratings of five dimensions of story quality (creativity, imagination, novelty, complexity, cohesiveness). We also analyzed the influence of four diverse categories of linguistic features (language productivity, lexical predictability, semantic similarity, and content composition) on these human ratings. Online storytellers improvised spoken stories, transcripts of which (N=108) were rated by both novices (N=304) and creative writing/storytelling experts (N=15) on the five quality dimensions above and analyzed in terms of the four linguistic feature categories above. Overall, experts and novices provided similar average ratings, and their rating spaces both showed two distinct underlying dimensions–concepts that are strongly correlated with creativity, and cohesiveness–which were, in turn, predicted by distinct sets of linguistic features. Notably, however, experts perceived the five dimensions as more differentiated, and their ratings were less driven by linguistic features than novices’. These findings indicate that experts’ judgments are more nuanced and may rely less on fine-grained linguistic and structural components of narratives, highlighting the importance of capturing individual differences and multidimensionality in the perception of stories.