Age-related differences in prefrontal glutamate are associated with increased working memory decay that gives the appearance of learning deficits

Article activity feed

1. 

   Version published to 10.7554/elife.85243 on eLife

   Apr 18, 2023
2. 

   eLife

   Apr 16, 2023

   Author Response

   Reviewer #1 (Public Review):

   This project aimed to understand if decision making impairments commonly observed in older adults arise from working memory (WM) or reinforcement learning (RL) deficits. Evidence in the paper suggests it is the former; they observe poorer task accuracy in older adults that is accompanied by a faster memory decay in older adults using a novel hierarchical instantiation of a previously validated computational model. There were no similar changes in RL in this model. These results are extended using Magnetic Resonance Spectroscopy (MRS) to measure glutamate and GABA levels in striatum, prefrontal and parietal regions. They found that impairments in working memory were linked to reductions of glutamate in PFC, particularly in the older adult group.

   The task employed is elegant and has been …

   Author Response

   Reviewer #1 (Public Review):

   This project aimed to understand if decision making impairments commonly observed in older adults arise from working memory (WM) or reinforcement learning (RL) deficits. Evidence in the paper suggests it is the former; they observe poorer task accuracy in older adults that is accompanied by a faster memory decay in older adults using a novel hierarchical instantiation of a previously validated computational model. There were no similar changes in RL in this model. These results are extended using Magnetic Resonance Spectroscopy (MRS) to measure glutamate and GABA levels in striatum, prefrontal and parietal regions. They found that impairments in working memory were linked to reductions of glutamate in PFC, particularly in the older adult group.

   The task employed is elegant and has been studied extensively in different populations and is well-validated (though here a hierarchical Bayesian extension is developed and validated). The results however may not be definitive in some respects; the paper did not replicate previously observed RL deficits. It therefore, remains possible that this is due to the sensitivity of the task to this RL component in ageing and future work is needed to fully bridge the gap in the literature.

   Thank you for the comment. If our understanding of the comment is correct, our results suggesting no impairments in the RL system conflict with previously observed RL deficits in older adults. In the introduction section, we discuss previous literature on RL deficits in old adults which yields largely mixed conclusions, wherein some experiments show RL impairments (Frank and Kong, 2008; Hämmerer et al., 2011; Samanez-Larkin et. al, 2014) and some do not (Grogan et al., 2019; Radulescu et al., 2016). Placing our experiment in the context of these mixed results, we aimed to use a task that addresses these inconsistencies, by reasoning that commonly used RL tasks and models do not account for additional processes that may contribute to learning (e.g. executive function/WM/attention), hence explaining why sometimes the deficits are observed and sometimes they are not. We can also point to our model parameter recovery (Appendix 1 - Figure 9), where we show that RL model parameters (e.g. learning rate) are successfully recovered - indicating that our model is sensitive to RL variability in participants, but we observe no differences split across age groups.

   Although the study is well-executed, there is an obvious limitation in the use of a cross-sectional design to address this question. The authors acknowledge this limitation in the discussion but could go further to highlight the potential confound of cohort effects on gaming, RL and WM tasks more generally. Without within-person change data, the evidence can only be suggestive of potential age-related decline. For this reason, it may be more appropriate to use the terminology "age-related differences' rather than "age-related declines" given the study design.

   Thank you for the comment. We have attempted to address the cohort effects by administering RBANS to old and young participants. Age-normed total RBANS (Randolph et al., 1998) scores were similar in both age groups (described in the first paragraph of the results section), which we took to suggest that our cohorts reflected comparable samples of the population with respect to overall cognitive ability. In addition, we show that certain aspects of performance (e.g. accuracy) decline within the group of older adults, and not just between the two groups, which would constitute an argument against cohort-based effects. We now elaborate further on the point of cross-sectional design in the discussion section on lines 410-417. As suggested by the reviewer, we have also adjusted the language throughout the manuscript to imply age-related differences instead of age-related decline.

   Reviewer #2 (Public Review):

   In this study, Rmus and colleagues contribute to the important open question of whether reinforcement learning deficits observed in older adults are due to impairments in basic learning processes, or can be attributed to a decline in working memory function. The authors present cross-sectional behavioral data from a task designed to assess the role of working memory in reinforcement learning. And they use computational modeling in conjunction with MR spectroscopy to demonstrate a relationship between prefrontal glutamate and age-related impairments in learning specific to working memory decay. I found the overall story compelling, the data novel, and the analysis carefully executed. Below I outline some areas in which the claims of the manuscript could be strengthened.

   1. I may have missed this, but does glutamate correlate with other model parameters? Or did the authors only focus on the WM parameters because of the age difference? In support of the specificity argument, it would be important to show that glutamate only predicts WM related parameters regardless of whether there was an age difference or not.

   Thank you for your suggestion. In Appendix 1-figure 7, we show correlations between glutamate and all model parameters. If glutamate captured impairments in RL computational processes, we would expect to see a correlation between glutamate and the learning rate. Below we show that glutamate does positively correlate with RL learning rate. However, there are parameter correlations within the model itself – making the direct correlations hard to interpret.To better understand the relationships between learning rate, working memory, and glutamate, we ran a model predicting MFG glutamate using all parameters that significantly correlated with MFG glutamate (MFG glutamate ~ 1 + learning rate + decay + omega3 + negative learning rate), and found that only WM decay predicted MFG glutamate when controlling for other factors (learning rate: t = -0.42, β = -.03, p =0.67; WM decay: t = -3.14, β = -0.30, p = .002; omega3: t = 1.84, β = .16, p = .07; negative learning rate: t = .56, β = .03, p = .57). Thus, while glutamate measures correlate with RL learning rate, these correlations seem to be driven by the fact that both glutamate and RL learning rate correlate with WM Decay. Note that negative learning rate influences both RL and WM processes’ updating (see computational modeling section), and thus cannot help us make claims about specificity of RL or WM mechanisms alone being related to glutamate.

   1. As it is somewhat common with these tasks, it seems like the model does not fully capture the performance deficit in OA (Fig. 2B), even when all the individual difference parameters in WM are allowed to vary. Can the authors say more about the discrepancy? This is an interesting datapoint which may give clues to mechanism.

   Thank you for your comment. We elaborated on this in detail in the Appendix 1 (Posterior predictive checks section). We have observed that in some blocks (particularly in ns=6 blocks), older adults only learned a correct response for a subset of the presented stimuli, and neglected to learn responses to other stimuli altogether. We have interpreted this as a possible strategy older adults used to reduce the difficulty of the ns=6 condition. This would explain the discrepancy between the data and the model predictions, as the model has no way of accounting for stimulus identity effects on learning (since the model predicts similar performance for all the stimuli). To test our reasoning, we have fit the model to a subset of data - excluding participants who have implemented this strategy, and predicted that this should reduce the model misfit. We found that this is indeed the case (Appendix 1 - Figure 4). This confirms that strategic prioritization of stimuli in some older adults negatively affected the fit of the model. While we believe that a better understanding of these contaminant response patterns in the RL-WM model is worthy of further investigation, we feel that it is beyond the scope of this paper, and might require task designs with even higher set sizes to elicit the strategic stimulus prioritization more robustly. We have now added a paragraph in the discussion to discuss this issue.

   1. Relatedly, it may not be possible with these data alone, but can authors discuss what the WM decay parameter captures? In particular for OA, the distinction between generating and maintaining a "task set" has been extensively written about. Older adults tend to have difficulty internally generating and flexibly deploying task sets, but somewhat paradoxically can perform better than YA in certain decision situations (e.g. when reward is dependent on previous choices, see Worthy et. Al. 2011). The task in this study necessarily pushes OA in a regime in which relying on familiar decision strategies is sub-optimal, and task sets must be continuously generated. Is there a type of intervention do authors expect would reverse the observed deficit in WM?

   In the RLWM model, WM stores stimulus-action-outcome weights. Using WM decay we can gradually reduce the stimulus-dependent weights on each trial where the stimulus is not observed (e.g. forgetting). These weights, therefore, get reduced with the rate of decay, by being pulled towards the uniform/uninformative values (1/nA, where nA is the number of actions) they were initialized to. It effectively captures forgetting of information with increased time delays (here time = number of intervening trials between successive stimulus presentations where the stimulus is not observed). It is possible that older adults might be prioritizing storage of different types of (irrelevant) task information (e.g. category of stimuli, or relationships between the stimuli), resulting in a tradeoff that might lead to faster decay in older adults, and that the younger adults neglect such information. This could also explain discrepancies between our model and older adults described above, as the model does not hold any assumptions about how stimulus identities might impact task performance strategy. If this was the case, if probed about such task-irrelevant prioritized information older adults could potentially perform better than younger adults (in a way that in the Worthy et al. (2011) paper the older adults perform better on a choice dependent task compared to younger adults). We are unable to test this idea in our dataset, but we believe that it could be a promising avenue for future research.

   1. There is a wealth of evidence suggesting striatal DA loss in older adults, which served as the basis for many of the original investigations and hypotheses regarding a simple RL deficit in OA (e.g. work by Shu-Chen Li and others). While the authors do not directly measure DA in this study, it would be helpful to place the results in the context of that literature.

   Thank you for pointing this out. In the introduction, we have discussed the mixed results from research on RL/dopamine deficits in older adults. Some of the literature suggests no impairments in striatal dopamine in older adults (Samanez-Larkin et. al, 2014; Bäckman et al., 2006), while some suggests absence of impairments (Grogan et al., 2019). Furthermore, while DA is important for RL updating, it is also potentially important for WM updating (O’Reilly and Frank, 2006), therefore a potential DA loss could affect both RL and WM, and not RL exclusively. Prior research also suggests that although correlative relationship between DA and cognitive functions has been recorded, the extent of generality/specificity of the effects of DA on cognition in aging (Bäckman et al., 2006), compared to resulting noise that impairs cognition (Li et al.,2001) should be studied more extensively in the future. We have not focused on dopamine in the study, but have now added a paragraph in the discussion section to address this on lines 402-407.

   1. Finally, the main argument of the paper as I read it is that PFC glutamate mediates the performance deficits observed in RL because it reflects a compromised WM system. Sample size permitting, it would be helpful to see a formal test of this mediation relationship.

   As highlighted in the response to the mediation point in essential revisions, we observe that glutamate mediates effect of WM on task performance, but that this mediation approach might be difficult to justify, due to WM decay and task performance having shared signal and noise (since WM decay is estimated from task performance). We have now included the mediation analysis in our Appendix 1 information and provided a conservative interpretation of it in the results section.

   Reviewer #3 (Public Review):

   Aging impacts many cognitive functions, and how these changes affect performance in different tasks is an important question. By testing 42 older and 36 younger healthy adults with a novel learning task and MR spectroscopy, Rmus et al addressed the important question whether age-related declines in learning are driven by WM, or by deficiencies of the RL system. The task varied the role of working memory in learning by asking participants to learn about either 3 or 6 stimulus response associations from feedback (set sizes 3 and 6). The paper combines a detailed computational account of participants behaviour and striatal and prefrontal/parietal MR spectroscopy in order to assess individual glutamate and GABA levels.

   The authors report an effect of set-size on learning in both are groups, and show that participant age is associated with (1) worse accuracy, (2) a larger set size performance difference, and (3) a heightened sensitivity to reward. Computational modeling showed that working memory decay differed between age groups, but that reliance on WM to perform the task at hand was similar in both age groups (similarly differing between conditions in both groups). Turning to the MRS results, the paper shows that an aggregate measure of glutamate relates to aggregate task performance, that prefrontal glutamate specifically relates to WM decay observed in the task, and that age was negatively associated with glutamate levels.

   While the paper is well worth reading and offers many interesting data points, the title's suggestion that "Age-related decline in prefrontal glutamate predicts failure to efficiently deploy working memory in working memory" is, in my opinion, not fully supported by the evidence. First, the authors don't report clear evidence for any age-related differences in WM reliance in the task overall. Second, the authors find that MFG glutamate relates significantly only to WM decay, not the parameter that captures WM deployment. Third, correlations don't imply predictive relations.

   We apologize for the lack of clarity in our wording. We agree that the title of the paper implies that the reliance on WM parameter differentiates older and young adults, while the results show that the difference is mostly captured by the WM decay parameter. We meant to communicate that the age-difference seems to be particularly rooted in the WM, but have chosen misleading/confusing words. We have proposed changing the title of the manuscript to “Age-related differences in prefrontal glutamate are associated with increased working memory decay that gives appearance of learning deficits” to minimize confusion. With regards to your last point, as outlined in our response to essential revisions, we agree that we should modify the language used in our manuscript to be more consistent with the associative rather than predictive nature of our results.

   Another important open question relates to the relatively large age difference in the effect of set-size on performance. The authors write that working memory will contribute less to performance in higher set size conditions. Yet, age differences are largest in the set size 6 condition, suggesting that RL-dependent learning is most severely impaired in learning (set size 6 performance), rather than WM dependent learning (set size 3 performance). Finally, a statistically significant age difference in reward sensitivity seems to be hardly integrated into the authors' overall interpretation.

   Working memory does contribute less in higher set-size condition; however, given the higher number of items, the delays between successive presentations of the stimuli in the high set-size condition are on average longer - which makes the effect of WM forgetting more pronounced. Furthermore, a WM impairment can have an indirect effect in RL, in that frequent failure to select correct action through WM leads to reduced ability to train RL on encoding correct responses (especially earlier in training, when the incremental RL hasn’t ‘caught up’ yet), and thus worse performance overall. As such, a larger effect of set size could potentially be indicative of either or both WM or RL process deficits. This most clearly underscores the importance of modeling - these complex interactions are difficult to intuit, but modeling allows us to establish cleaner mechanistic explanations of observed behavioral patterns/group performance deficits (e.g. while on the surface impairment might look to be RL driven, it is actually better explained by a WM parameter, such as WM decay in older adults - this can). With regards to reward sensitivity, the same explanation applies - there are multiple mechanisms through which differences in reward sensitivity could occur (e.g. slower learning rate, or increased RL recruitment due to failure of WM), which further emphasizes the need for modeling.

   In short, in a complex task, there are often multiple ways to explain the same qualitative feature and here we have leaned on computational modeling to identify the computational elements that differed across groups. However we have now also simulated data from our computational models using posterior predictive checks to show that they can reproduce core descriptive features of the original data, including those noted above, and to examine the degree to which different features can be mapped onto the working memory decay parameter (Appendix 1 Figure 5).

   Read the original source
3. 

   eLife

   Apr 15, 2023

   eLife assessment

   This important study combines behavior, computational modelling and magnetic resonance spectroscopy to address the question whether age-related declines in learning are driven by declines in working memory or deficiencies of the RL system. The general approach is solid, but the presented evidence to support the papers' main claims could be stronger. With additional analyses and adaptation of the main claims, the paper could be of high interest for researchers in the field of cognitive aging and decision making.

   Read the original source
4. 

   eLife

   Apr 15, 2023

   Reviewer #1 (Public Review):

   This project aimed to understand if decision making impairments commonly observed in older adults arise from working memory (WM) or reinforcement learning (RL) deficits. Evidence in the paper suggests it is the former; they observe poorer task accuracy in older adults that is accompanied by a faster memory decay in older adults using a novel hierarchical instantiation of a previously validated computational model. There were no similar changes in RL in this model. These results are extended using Magnetic Resonance Spectroscopy (MRS) to measure glutamate and GABA levels in striatum, prefrontal and parietal regions. They found that impairments in working memory were linked to reductions of glutamate in PFC, particularly in the older adult group.

   The task employed is elegant and has been studied extensively in …

   Reviewer #1 (Public Review):

   This project aimed to understand if decision making impairments commonly observed in older adults arise from working memory (WM) or reinforcement learning (RL) deficits. Evidence in the paper suggests it is the former; they observe poorer task accuracy in older adults that is accompanied by a faster memory decay in older adults using a novel hierarchical instantiation of a previously validated computational model. There were no similar changes in RL in this model. These results are extended using Magnetic Resonance Spectroscopy (MRS) to measure glutamate and GABA levels in striatum, prefrontal and parietal regions. They found that impairments in working memory were linked to reductions of glutamate in PFC, particularly in the older adult group.

   The task employed is elegant and has been studied extensively in different populations and is well-validated (though here a hierarchical Bayesian extension is developed and validated). The results however may not be definitive in some respects; the paper did not replicate previously observed RL deficits. It therefore, remains possible that this is due to the sensitivity of the task to this RL component in ageing and future work is needed to fully bridge the gap in the literature.

   Although the study is well-executed, there is an obvious limitation in the use of a cross-sectional design to address this question. The authors acknowledge this limitation in the discussion but could go further to highlight the potential confound of cohort effects on gaming, RL and WM tasks more generally. Without within-person change data, the evidence can only be suggestive of potential age-related decline. For this reason, it may be more appropriate to use the terminology "age-related differences' rather than "age-related declines" given the study design.

   Read the original source
5. 

   eLife

   Apr 15, 2023

   Reviewer #2 (Public Review):

   In this study, Rmus and colleagues contribute to the important open question of whether reinforcement learning deficits observed in older adults are due to impairments in basic learning processes, or can be attributed to a decline in working memory function. The authors present cross-sectional behavioral data from a task designed to assess the role of working memory in reinforcement learning. And they use computational modeling in conjunction with MR spectroscopy to demonstrate a relationship between prefrontal glutamate and age-related impairments in learning specific to working memory decay. I found the overall story compelling, the data novel, and the analysis carefully executed. Below I outline some areas in which the claims of the manuscript could be strengthened.

   1. I may have missed this, but does …

   Reviewer #2 (Public Review):

   In this study, Rmus and colleagues contribute to the important open question of whether reinforcement learning deficits observed in older adults are due to impairments in basic learning processes, or can be attributed to a decline in working memory function. The authors present cross-sectional behavioral data from a task designed to assess the role of working memory in reinforcement learning. And they use computational modeling in conjunction with MR spectroscopy to demonstrate a relationship between prefrontal glutamate and age-related impairments in learning specific to working memory decay. I found the overall story compelling, the data novel, and the analysis carefully executed. Below I outline some areas in which the claims of the manuscript could be strengthened.

   1. I may have missed this, but does glutamate correlate with other model parameters? Or did the authors only focus on the WM parameters because of the age difference? In support of the specificity argument, it would be important to show that glutamate only predicts WM related parameters regardless of whether there was an age difference or not.
   2. As it is somewhat common with these tasks, it seems like the model does not fully capture the performance deficit in OA (Fig. 2B), even when all the individual difference parameters in WM are allowed to vary. Can the authors say more about the discrepancy? This is an interesting datapoint which may give clues to mechanism.
   3. Relatedly, it may not be possible with these data alone, but can authors discuss what the WM decay parameter captures? In particular for OA, the distinction between generating and maintaining a "task set" has been extensively written about. Older adults tend to have difficulty internally generating and flexibly deploying task sets, but somewhat paradoxically can perform better than YA in certain decision situations (e.g. when reward is dependent on previous choices, see Worthy et. Al. 2011). The task in this study necessarily pushes OA in a regime in which relying on familiar decision strategies is sub-optimal, and task sets must be continuously generated. Is there a type of intervention do authors expect would reverse the observed deficit in WM?
   4. There is a wealth of evidence suggesting striatal DA loss in older adults, which served as the basis for many of the original investigations and hypotheses regarding a simple RL deficit in OA (e.g. work by Shu-Chen Li and others). While the authors do not directly measure DA in this study, it would be helpful to place the results in the context of that literature.
   5. Finally, the main argument of the paper as I read it is that PFC glutamate mediates the performance deficits observed in RL because it reflects a compromised WM system. Sample size permitting, it would be helpful to see a formal test of this mediation relationship.

   Read the original source
6. 

   eLife

   Apr 15, 2023

   Reviewer #3 (Public Review):

   Aging impacts many cognitive functions, and how these changes affect performance in different tasks is an important question. By testing 42 older and 36 younger healthy adults with a novel learning task and MR spectroscopy, Rmus et al addressed the important question whether age-related declines in learning are driven by WM, or by deficiencies of the RL system. The task varied the role of working memory in learning by asking participants to learn about either 3 or 6 stimulus response associations from feedback (set sizes 3 and 6). The paper combines a detailed computational account of participants behaviour and striatal and prefrontal/parietal MR spectroscopy in order to assess individual glutamate and GABA levels.

   The authors report an effect of set-size on learning in both are groups, and show that …

   Reviewer #3 (Public Review):

   Aging impacts many cognitive functions, and how these changes affect performance in different tasks is an important question. By testing 42 older and 36 younger healthy adults with a novel learning task and MR spectroscopy, Rmus et al addressed the important question whether age-related declines in learning are driven by WM, or by deficiencies of the RL system. The task varied the role of working memory in learning by asking participants to learn about either 3 or 6 stimulus response associations from feedback (set sizes 3 and 6). The paper combines a detailed computational account of participants behaviour and striatal and prefrontal/parietal MR spectroscopy in order to assess individual glutamate and GABA levels.

   The authors report an effect of set-size on learning in both are groups, and show that participant age is associated with (1) worse accuracy, (2) a larger set size performance difference, and (3) a heightened sensitivity to reward. Computational modelling showed that working memory decay differed between age groups, but that reliance on WM to perform the task at hand was similar in both age groups (similarly differing between conditions in both groups). Turning to the MRS results, the paper shows that an aggregate measure of glutamate relates to aggregate task performance, that prefrontal glutamate specifically relates to WM decay observed in the task, and that age was negatively associated with glutamate levels.

   While the paper is well worth reading and offers many interesting data points, the title's suggestion that "Age-related decline in prefrontal glutamate predicts failure to efficiently deploy working memory in working memory" is, in my opinion, not fully supported by the evidence. First, the authors don't report clear evidence for any age-related differences in WM reliance in the task overall. Second, the authors find that MFG glutamate relates significantly only to WM decay, not the parameter that captures WM deployment. Third, correlations don't imply predictive relations.

   Another important open question relates to the relatively large age difference in the effect of set-size on performance. The authors write that working memory will contribute less to performance in higher set size conditions. Yet, age differences are largest in the set size 6 condition, suggesting that RL-dependent learning is most severely impaired in learning (set size 6 performance), rather than WM dependent learning (set size 3 performance). Finally, a statistically significant age difference in reward sensitivity seems to be hardly integrated into the authors' overall interpretation.

   The issues laid out above set aside, the paper has the potential to make an important contribution to the literature on cognitive aging.

   Read the original source
7. 

   Version published to 10.1101/2022.11.28.518237v2 on bioRxiv

   Apr 10, 2023
8. 

   Version published to 10.1101/2022.11.28.518237v1 on bioRxiv

   Nov 29, 2022