Task-specific representational spaces for memory formation in the medial temporal lobe
Curation statements for this article:-
Curated by eLife
eLife Assessment
This important study provides novel insights into how the medial temporal lobe and visual cortex flexibly represent concepts according to current task demands. The authors developed an ingenious experimental design to provide compelling evidence that patterns of activity in memory-related brain regions, particularly the hippocampus, reflect conceptual relationships relevant to the current task. Some additional analyses could further clarify the nature of the task-specific effects and how they relate to memory.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Adaptive behavior requires stimuli to be processed in a way that meets the requirements of the current task. This is especially relevant for natural concepts that comprise a multitude of representational formats. The medial temporal lobe (MTL) has been proposed to support such flexibility by organizing stimuli within relational representational spaces serving as cognitive maps. However, while various studies investigated the formation of novel cognitive maps based on abstract features, it remains unclear how our vast existing conceptual knowledge is recruited into task-relevant representational spaces and whether these spaces influence the formation of novel episodic memories. Here we show that neural representations of natural concepts in the MTL and neocortex are embedded in low-dimensional representational spaces of task-relevant features that flexibly reorganize when task demands change. Crucially, hippocampal pattern similarity reflects task-relevant, but not task-irrelevant conceptual feature dimensions of the same set of natural concepts, indicating context-dependent reconfiguration of representational spaces. Euclidean distances within and across these spaces predicted subsequent memory confidence for individual items, suggesting a putative metric of task-dependent neural distinctiveness for the formation of memory traces. Finally, we dissociate task- and stimulus-dependent formats in neocortex, demonstrating complementary representations of natural stimuli that differentially contribute to memory formation. Our findings suggest that relational coding of task-relevant features in the MTL may serve as an index of representational distinctiveness that contributes to the formation of novel episodic memories and integrate MTL accounts based on spatial coding and episodic memory.
Article activity feed
-
eLife Assessment
This important study provides novel insights into how the medial temporal lobe and visual cortex flexibly represent concepts according to current task demands. The authors developed an ingenious experimental design to provide compelling evidence that patterns of activity in memory-related brain regions, particularly the hippocampus, reflect conceptual relationships relevant to the current task. Some additional analyses could further clarify the nature of the task-specific effects and how they relate to memory.
-
Reviewer #1 (Public review):
Rau et al. ask whether stimulus representations change as a function of task demands, and whether properties of these task-specific representational spaces relate to the formation of episodic memories. Participants are presented with pairs of animal stimuli and rate their relationship along two pairs of feature dimensions (e.g., Size & Activity versus Proximity & Climate). Each pair defines a context. The authors reconstruct participant-specific representational spaces and investigate how their geometry relates to multivoxel activity patterns in the medial temporal lobe, how these representations reorganise across task contexts, and how neural representations relate to subsequent episodic memory.
The paper asks whether existing conceptual knowledge representations in the MTL are dynamically reorganised …
Reviewer #1 (Public review):
Rau et al. ask whether stimulus representations change as a function of task demands, and whether properties of these task-specific representational spaces relate to the formation of episodic memories. Participants are presented with pairs of animal stimuli and rate their relationship along two pairs of feature dimensions (e.g., Size & Activity versus Proximity & Climate). Each pair defines a context. The authors reconstruct participant-specific representational spaces and investigate how their geometry relates to multivoxel activity patterns in the medial temporal lobe, how these representations reorganise across task contexts, and how neural representations relate to subsequent episodic memory.
The paper asks whether existing conceptual knowledge representations in the MTL are dynamically reorganised depending on task demands. This is an important and well-motivated question. By leveraging participants' existing knowledge about animals together with a clever within-stimulus manipulation of task context, the study provides a relatively direct test of whether representational geometry flexibly reorganises with behavioural relevance
The analyses are executed carefully. For example, the reconstruction of representational spaces is validated using leave-one-pair-out cross-validation, and the critical neural analyses are restricted to inferred concept pairs. The relationship between behaviourally reconstructed spaces and neural representations therefore cannot simply be explained by the specific comparisons participants made during the task. Potential confounds, such as perceptual similarity and mean BOLD activity, are explicitly controlled for in the neural similarity analyses.
Overall, I really like this paper. The experimental design and the analyses are very thorough, and I find much of the data convincing. I have a few suggestions that I think could be addressed to improve the paper:
(1) There is no temporal gap between the second stimulus presentation and the rating period. Because the BOLD response is slow, responses elicited by the second image, stimulus comparison and evaluation, response preparation, and potentially the motor response will overlap substantially. I am therefore concerned that an RSA effect attributed to a "task-specific representation of the concept during image viewing" might partly reflect the ensuing relational evaluation process. Could the authors clarify how well the image and rating event regressors can be separated? I would also suggest testing the RSA effect using the first images alone, because these are temporally better separated from the rating period. If the task-specific RSA effect holds for these images, the claim of task-dependent representational geometry during stimulus viewing would become considerably stronger. If this is not the case, I suggest toning down statements such as "task-relevant information affected representations during the viewing of the images, before behavioral ratings were obtained".
(2) I am uncertain about the concept-level "remapping" measure. The authors define it as the difference in radial distance from the origin between space 1 and space 2. Firstly, could the authors clarify whether the concept-level reorganisation measure is the signed difference between distances to the origin or the absolute difference? The Methods and Figure 4 caption specify dist(space1) - dist(space2), whereas the exclusively positive values in Figure 4B and the interpretation in terms of the magnitude of reorganisation suggest that the absolute difference may have been used.
(3) More generally, I am wondering whether a change in radial distance really measures how much a concept moves between two spaces. For example, a concept could have the same distance from the origin in both spaces while having very different relationships to the other concepts. Measuring only changes in eccentricity would show no remapping in such a case. One possible alternative would be to characterize each concept by its vector of distances to all other concepts and quantify how much this relational profile changes between spaces, for example:
concept remapping = 1 − corr(distances to all other concepts in space 1, distances to all other concepts in space 2)
The less similar the two relational-distance profiles are, the greater the remapping. Could the authors show whether the results hold with this or another measure that captures changes in relational embedding beyond changes in eccentricity?
(4) The two images are encoded under quite different circumstances: the second image is immediately followed by the rating process, whereas the first image is not. The authors show that larger 2D distances predict higher subsequent memory confidence for first and second images (although the effect seems to be more pronounced for second images), but they do not report whether memory itself was better for first versus second images, nor whether the distance effect differs significantly between positions. This could be informative to know. Furthermore, the analyses relating representational distance to memory confidence appear to be restricted to subsequently remembered stimuli. Does this relationship also hold when all encoded items are included? This would more directly test whether representational distance predicts subsequent memory strength, rather than "only" confidence among images that were successfully recognised.
-
Reviewer #2 (Public review):
In this study, Rau and colleagues investigate how concept knowledge is organized into task-specific representational spaces in the MTL, and how this relates to episodic memory formation. Using behavioral ratings of animal concepts in two task contexts, they reconstruct conceptual spaces and show that MTL, and particularly HPC, representations align with task-relevant spaces, track their reorganization across tasks, and relate to subsequent memory. This is an interesting and well-designed study that addresses important questions. The methods are careful, and testing generalization via inferred concept pairs is an elegant approach.
I have a number of questions, including clarifications and questions about the analyses. These concern what some of the metrics capture and how they relate to the authors' …
Reviewer #2 (Public review):
In this study, Rau and colleagues investigate how concept knowledge is organized into task-specific representational spaces in the MTL, and how this relates to episodic memory formation. Using behavioral ratings of animal concepts in two task contexts, they reconstruct conceptual spaces and show that MTL, and particularly HPC, representations align with task-relevant spaces, track their reorganization across tasks, and relate to subsequent memory. This is an interesting and well-designed study that addresses important questions. The methods are careful, and testing generalization via inferred concept pairs is an elegant approach.
I have a number of questions, including clarifications and questions about the analyses. These concern what some of the metrics capture and how they relate to the authors' conclusions.
(1) On inferred distances / latent spaces:
The RSA is restricted to inferred (never directly rated) concept pairs, which seems an important feature of the design: regions aligned with these distances represent a task-relevant space that generalizes beyond the explicitly probed comparisons. This could be emphasized more. Relatedly, I was surprised not to see frontoparietal or "multiple-demand" regions, which might be expected to represent task-relevant comparisons. One possibility is that such regions encode the explicitly compared pairs (external attention), whereas HPC and default-mode regions encode the latent space supporting inference (internal attention). Could the authors test this by, e.g., comparing alignment for rated vs inferred pairs?
(2) Sizes or strength of the effects:
Inferring distances is a nice approach, and the cross-validation is a good way to test whether the reconstructed spaces are valid. Is there a sense of prediction accuracy, for example, the error and variance for a given inferred distance? A standardized measure might be more interpretable.
Similarity is greater for task-relevant than task-irrelevant spaces. Are the absolute values interpretable, or do they depend on how participants used the rating scale? In any case, a noise ceiling or effect size measure would help indicate how strong these effects are.
(3) Space-level reorganization (Figure 4A):
What does the space-level reorganization analysis (Figure 4A) show? As I understand it, space1-space2 similarity is computed separately for the reconstructed behavioral spaces and the HPC concept RSMs, and the two values are then correlated across participants. It's not clear to me why this between-participant relationship would indicate that the HPC flexibly reorganizes representations according to task demands. The subsequent within-subject concept-level analysis seems to capture this better?
(4) Concept-level reorganization metric:
Could the authors clarify how the origin (0, 0) of the reconstructed spaces is defined? Since the least-squares solution is only determined up to a constant shift, I assume the origin corresponds to the centroid of the concepts, but it would help to state this. It would also be useful to confirm whether dimensions were scaled (e.g., z-scored) before computing distances. Otherwise, differences in rating spread between spaces could influence the metric.
More generally, I wondered whether the metric captures reorganization per se, or rather a change in how extreme or atypical a concept is relative to the others. A concept could change its relationships to other concepts substantially while keeping a similar distance from the centroid.
This also seems different from the neural measure, which reflects how much a concept's own pattern changes across spaces. Would it be more straightforward to compare each concept's pairwise distances to the other concepts across spaces? This would also relate more clearly to the neural data, for example via the equivalent neural similarity profiles.
Finally, the hypothesis implies greater neural differentiation across than within spaces for concepts that reorganize more. Between- and within-space similarity are currently tested separately, but a significant effect for one and not the other does not show that they differ. Would it be informative to test a per-concept differentiation score (within minus between) directly?
(5) Memory confidence:
Why is position in the space predicted to relate to memory confidence rather than to memory performance itself? I would have predicted an effect on memory, since a concept that is more distinctive in this space should be better remembered. I would have predicted this even more so for sensory features.
(6) Anterior-posterior HPC:
Is there an anterior-posterior HPC difference for more abstract vs more concrete conceptual features, and potentially in other areas? For example, size is relatively concrete, whereas proximity and climate are more abstract or less tied to perceptual features (though I'm not sure of the exact ranking).
(7) Voxel overlap analysis:
The overlap analysis (Figure 6D-E) compares voxels within TFCE-significant clusters across contrasts. However, cluster extent depends on the sensitivity of each contrast rather than on the tuning or strength of individual voxels. For example, the task-specific map includes very large clusters, and the DNN map combines eight layers. Absence of overlap also does not imply absence of a representation, since voxels may carry information below threshold. The percentages may therefore be misleading. Could the authors use a threshold-independent approach, such as correlating unthresholded maps or comparing effect sizes within ROIs?
-
Reviewer #3 (Public review):
Summary:
The authors investigate whether conceptual representations in the brain, and in particular in the medial temporal lobe, flexibly adapt to current task demands, and whether these representations shape memory formation. Participants rated pairs of animal images on two conceptual dimensions during fMRI, each in two distinct two-dimensional spaces. From these ratings, the authors reconstructed participant-specific cognitive maps and related them to neural pattern similarity. Neural similarity in the hippocampus and perirhinal cortex reflected distances in the task-relevant space, and representations in the medial temporal lobe and across ventral and dorsal visual regions were more aligned with the task-relevant than the task-irrelevant space. In a surprise recognition test, representational …
Reviewer #3 (Public review):
Summary:
The authors investigate whether conceptual representations in the brain, and in particular in the medial temporal lobe, flexibly adapt to current task demands, and whether these representations shape memory formation. Participants rated pairs of animal images on two conceptual dimensions during fMRI, each in two distinct two-dimensional spaces. From these ratings, the authors reconstructed participant-specific cognitive maps and related them to neural pattern similarity. Neural similarity in the hippocampus and perirhinal cortex reflected distances in the task-relevant space, and representations in the medial temporal lobe and across ventral and dorsal visual regions were more aligned with the task-relevant than the task-irrelevant space. In a surprise recognition test, representational distinctiveness along the visual hierarchy predicted subsequent memory.
Strengths:
This is a timely study that brings together work on conceptual knowledge, cognitive maps, and episodic memory. I found the design particularly clever. Because each participant rated the same concepts in two spaces that share no dimensions, task effects can be tested within participants, and the use of a new exemplar on every presentation makes it possible to test memory for individual images. The reconstruction of the full representational spaces from a limited number of ratings is a nice solution, and restricting the neural analyses to pairs that were never directly rated is a good way to test whether the representations go beyond the comparisons participants actually made. I also appreciated the control for mean activation and for perceptual similarity across all layers of the DNN, which makes it more likely that the effects are not simply visual.
Weaknesses:
(1) The authors write that RTs were predicted by the currently rated dimension "but not by the other feature dimension". However, the other dimension also has a significant effect (P = 0.001), and the positive Z suggests that RTs were longer when animals differed more on the unrated dimension. So the rated dimension has a stronger influence, but it is not specific, and the text should be changed accordingly. Also, the Z and χ² values for this effect do not seem to match (Z² ≈ 534, but χ² = 10.89). Please double-check.
(2) I am not fully convinced by the subtraction used to get the task-specific effect, especially looking at the MTL results. In PRC and PHC, there is a significant negative relation with the task-irrelevant space. In PHC, the task-relevant correlation is not significant (P = 0.075), so the task-specific effect in that region seems to come mostly from the negative irrelevant correlation, not from the relevant space. Something similar may be happening in PRC. If so, "task-specific" means something different from what the reader would assume, and this deserves some discussion. Could the two reconstructed maps be correlated within participants? In that case, fitting one map would partly mean not fitting the other, and the subtraction would not tell us what each map explains on its own. It would be useful to report the correlation between the space1 and space2 distance matrices for each participant, and to check the task-specific effects with multiple regression (both matrices as predictors) or partial correlation.
(3) The voxel-overlap analysis (lines 441-458) is interesting, but I think the conclusions go a bit beyond the data. Some points:
(a) How many voxels survive the threshold depends a lot on the power of each analysis, so the 66% of task-specific voxels does not necessarily mean "broad spatial tuning towards task-relevant feature representations".
(b) The binomial test assumes voxels are independent, which is hard to justify with smoothed data and searchlight spheres. I am also not sure a uniform distribution across formats is a meaningful null.
(c) The overlap with SME voxels is reported without a chance level. Something like a spatial permutation, taking map size and autocorrelation into account, would make these numbers easier to interpret.
(d) If overlap is expressed relative to map size, Fc has the highest overlap with SME (58% of Fc voxels) and the task-specific format the lowest (18%). This seems to go against the idea that task-relevant formats are preferentially encoded, no?
-
-
-
-
Authors published a paper in Biorxiv, they used python-bx in the study. Including #RRIDs will make this less ambiguous. SciScore made a table with this resource, see “Automated Services” module (download as csv, xml or #jats) #OpenScience #reproducibility
-
Task-relevant cognitive maps in episodic memory https://www.biorxiv.org/content/10.1101/2025.05.23.655754v1
-
Task-relevant cognitive maps in episodic memory https://www.biorxiv.org/content/10.1101/2025.05.23.655754v1
-