Measuring Naturalistic Speech Comprehension in Real Time

Speech comprehension has been described as an effortless and robust process; yet, in real-world contexts, it is common for a listener to misunderstand what was said, or fail to derive meaning entirely. Typically, methods of measuring speech comprehension are applied `post-hoc' - that is, after the comprehension has happened. This lacks the time resolution necessary to capture comprehension as it dynamically unfolds, and introduces memory confounds. The absence of appropriate time-resolved behavioral measurements has limited the field's understanding of the neural processes involved in online comprehension. To overcome these challenges, we designed and tested a novel way of measuring real-time speech comprehension during naturalistic listening. We built a slider device that synchronizes with behavioral and neural recording software and provides millisecond read-out. Thirty participants listened to audiobook segments while providing continuous comprehension ratings using the slider. To vary comprehension success, we presented speech segments at speed factors 1-5 times faster than normal. We validated the time-resolved slider data against established speech comprehension assessment methods: (i) 10-point scale rating, (ii) multiple choice question, and (iii) written summary, and obtained working memory and auditory acuity performance to account for by-participant variability that is not related to comprehension. Overall, our findings validate our novel time-resolved comprehension measure and demonstrate that it is possible to derive an online behavioral measure of real-time speech comprehension, the lack of which has been a major limitation of naturalistic language studies. We also confirmed numerous limitations of static post-hoc assessments, including challenges of multi-choice question design, and the confounding of potential effects due to recency bias and comprehension for summarization. The measure proposed here overcomes the constraints of static post-hoc assessments and can be effectively integrated with neuroimaging techniques, offering a valuable tool for future research on dynamic processes during naturalistic listening.