Temporal asymmetry of neural representations predicts memory decisions

A stimulus can be familiar for multiple reasons. It might have been recently encountered, or is similar to recent experience, or is similar to ‘typical’ experience. Understanding how the brain translates these sources of similarity into memory decisions is a fundamental, but challenging goal. Here, using fMRI, we computed neural similarity between a current stimulus and events from different temporal windows in the past and future (from seconds to days). We show that trial-by-trial memory decisions (is this stimulus ‘old’?) were predicted by the difference in similarity to past vs. future events (temporal asymmetry). This relationship was (i) evident in lateral parietal and occipitotemporal cortices, (ii) strongest when considering events from the recent past (minutes ago), and (iii) most pronounced when veridical (true) memories were weak. These findings suggest a new perspective in which the brain supports memory decisions by comparing what actually occurred to what is likely to occur.