Estimating the reliability of the lure discrimination index for studying brain-behavior correlations and individual differences in memory

There is a growing interest in psychological tasks that closely correspond to specific neural computations for studying brain-behavior associations across development and in disease. The psychometric properties of such tasks, particularly their reliability, are critical when designing correlational research in longitudinal, clinical, or other studies. The Mnemonic Similarity Task (MST) is a widely used paradigm for assessing the ability to discriminate between similar memory representations. Given its reliance on hippocampal pattern separation, the MST has been increasingly employed as an index of hippocampal function in cognitive and clinical neuroscience. Here, we evaluate the reliability of the Lure Discrimination Index–the most common performance metric of the MST–to determine its suitability for these research contexts. First, in a sample of 96 younger (20–31 years) and 109 older (66–83 years) participants, we estimate the consistency reliability of the MST using permutation-based methods. Our findings indicate that while consistency reliability estimates range from moderate to high, they are contingent on sample heterogeneity, and the resulting effect size attenuation is dependent on the study context. Second, using a subsample of 49 participants (13 younger and 36 older adults) who completed the MST twice with a one-year interval, we calculated the absolute agreement reliability with intraclass correlations. We found low to moderate stability of individual scores over time and tentative evidence for the presence of practice effects in the younger sample. We discuss the implications for correlational research and studies where score stability is important. Further, we provide tools to inform future research design.