Classification of Decomposed Neural Data in Memory Networks and LLM-Based Stimuli Processing

Naturalistic paradigms, where participants are exposed to real-world stimuli (e.g., narratives) are an important technique in memory research. They may provide a more complete assay of human memory processes and their underlying mechanisms. In these procedures, stimuli are continuous and do not have well-defined scene boundaries. Defining scene boundaries based on narrative events such as plot twists or character developments is crucial as these moments are known to influence memory recall. Aligning neural activity with such boundaries helps in studying the dynamics of memory recall during narrative experiences. However, segmenting narratives based on memory-driven cues is difficult due to the continuous flow of events. To overcome this, we developed an automatic scene segmentation technique using large language models (LLMs). The LLMs segment narratives into meaningful scenes offering a consistent unbiased method for recall-based segmentation. These segments are then used to analyze brain dynamics from fMRI data. In our study, 180 participants listened to four different stories while undergoing fMRI. Functional connectivity (FC) was computed based on the LLM-derived scene segments. For memory recall analysis, classification models were trained using participants’ recall scores as labels. To further understand the neural basis of memory, FC matrices were decomposed into shared (low-rank) and individual (idiosyncratic) components. Classification results demonstrate that LLM-based segmentation effectively defines scene boundaries and that memory recall is not solely tied to common or idiosyncratic activity. This approach offers a robust framework for exploring brain-behavior relationships in naturalistic memory research.