Reinstatement of cortical outcome representations during higher-order learning

  1. Lennart Luettgau
  2. Emanuele Porcu
  3. Claus Tempelmann
  4. Gerhard Jocham

  • Curated by eLife

    eLife logo

    Summary: All reviewers agreed that the neural mechanisms by which value is conferred to stimuli that were never directly paired with reinforcement is an important topic. However, individual reviewers raised questions regarding the study design and data analysis. In particular, reviewers agreed it was not clear how you could distinguish BOLD responses to CS1 and CS2 given the temporal proximity of their presentation. They also wondered whether the current results would provide enough advance beyond previous work.

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Naturalistic learning scenarios are characterized by infrequent experience of external feedback to guide behavior. Higher-order learning mechanisms like second-order conditioning (SOC) may allow stimuli that were never experienced together with reinforcement to acquire motivational value. Despite its explanatory potential for real-world learning, surprisingly little is known about the neural mechanism underlying such associative transfer of value in SOC. Here, we used multivariate cross-session, cross-modality searchlight classification on functional magnetic resonance imaging data obtained from humans during SOC. We show that visual first-order conditioned stimuli (CS) reinstate cortical patterns representing previously paired gustatory outcomes in the lateral orbitofrontal cortex (OFC). During SOC, this OFC region showed increased functional covariation with amygdala, where neural pattern similarity between second-order CS and outcomes increased from early to late stages of SOC. Our data suggest a mechanism by which motivational value is conferred to stimuli that were never paired with reinforcement.

Article activity feed

  1. eLife

    Reviewer #3:

    This paper shows that during a second-order conditioning (SOC) task, the representation of a conditioned outcome is represented in the lateral orbitofrontal cortex (lOFC). The BOLD signal in this region shows increased functional coupling with the amygdala for second-order conditioned stimuli that indirectly predict a negative outcome. The authors suggest these findings reflect a mechanism by which value is conferred to stimuli that were never paired with reinforcement.

    The paper tackles an interesting question concerning the neural mechanisms that support second order conditioning. The task design includes relevant controls and, on the whole, the findings support the claims made by the authors. I have a few questions about interpretation of the data, but my main suggestion would be to revise the framing of the article. There are many previous studies that have investigated the mechanisms that support second order conditioning which are not always given due credit. I believe this paper would benefit from placing the hypotheses and findings more firmly within the context of previous literature.

    Comments:

    1. The authors test the hypothesis that CS2 is directly paired with a neural representation of the US. They state that this hypothesis 'has never been tested to date'. However, a number of studies have shown evidence for and against this hypothesis (for example: Wimmer and Shohamy 2012; Wang et al., 2020; Barron et al., 2020). Can the authors clarify how the hypothesis tested here differs from those investigated previously? In addition, it is not clear to me how the four potential mechanisms they propose are really distinct from each other?

    2. Relatedly, given the authors use an SOC paradigm that differs from sensory preconditioning studies used by many previous authors, does the difference in task paradigm provide new insight? Do the authors expect the neural mechanism to be the same or different between their version of SOC and sensory preconditioning?

    3. Why is the behavioural data in Figure 1F bimodal for CS1 and CS2? i.e. what does choice probability of 0 for CS2+ vs CS2- mean for a given subject?

    4. To test the author's hypothesis, is it not necessary to assess evidence for US in response to CS2? They instead report reactivation of US in response to CS1 and for the PPI it is not clear to me how the authors distinguish between CS1 and CS2 given the temporal proximity in their presentation (Figure 1D).

    5. For the PPI, is there a main effect of CS- and CS+ versus CSn in lOFC? If not, how does this affect interpretation of the PPI? On a separate note, is the effect reported in Figure 3 really in the hippocampus? Does it survive small volume correction using a hippocampal mask?

    6. The following is stated as a premise: "To form an associative link between CS2 and US, the reinstated US patterns need to be projected from their cortical storage site to regions like amygdala and hippocampus, allowing for convergence of US and CS2 information." This potentially seems fair for the hippocampus, with added reference to relevant literature (e.g. publications from Shohamy and Preston labs), but in my opinion the jury is still out on this one. It is not clear to me why we necessarily expect amygdala here.

    7. There are various strong statements that in my opinion need to be toned down in light of existing literature. For example, the paper claims this study is the first to show evidence for implicit inference. However, as far as I'm aware, Wimmer & Shohamy 2012 also found no evidence for explicit memory of stimulus-stimulus associations with no relationship between measures of explicit memory and decision bias. Similarly, the authors claim this paper is 'the only report so far of behavioral evidence for associative transfer of motivational value during human second-order conditioning', overlooking a large number of other studies that have shown similar behavioural effects.

  2. eLife

    Reviewer #2:

    The authors investigate the neural correlates of second order conditioning in carefully designed behavioural experiments coupling multivariate fMRI and functional connectivity. They found that the lateral OFC in connection with the amygdala, plays an important role. I think the paper represents a valuable addition to the human cognitive literature, where second order conditioning is surprisingly under-investigated. I have only a few suggestions to make.

    I encourage the authors to complement the multivariate analyses with a standard univariate analysis. To be clear, I am not without seeing the added value of the multivariate approach, however, given the extensive literature on the neural bases of conditioning using univariate analyses and the strong prediction about directionally of the effects in the OFC (which should positively encoded expected values and rewards), I think the paper would definitely benefit from including the univariate results for the main contrasts / variables.

    I am also curious to see the reaction times in the attentional control task analyzed to check if they were affected by the underlying conditioning procedure. Following the Pavlovian-to-Instrumental transfer theory, we should observe that the reaction times are slower for negative (aversive) stimuli and faster for positive (appetitive) stimuli.

  3. eLife

    Reviewer #1:

    This manuscript by Luettgau et al. describes a study of second-order conditioning in humans. The behavioral task involved visual first- (CS1) and second-order cues (CS2) and gustatory outcomes (US). Behavioral results show that subjects preferred both the CS1+ and CS2+ over the CS1- and CS2-, respectively. MVPA shows that the CS1 evokes US representations in the lateral OFC, and that US representations in the amygdala increase over second-order conditioning. This study addresses an important and novel question. However, I have several major concerns regarding the study design and data analysis:

    1. I do not see how it would be possible to disentangle responses to the CS1 and CS2 in this task. The delay between the CS2 and CS1 is only 500 ms, which is not long enough to disentangle fMRI responses to the two CS.

    2. For the main "reinstatement" analysis, activity was averaged across both CS2 and CS1, and so it is unclear whether reinstatement is driven by the CS1 or CS2. The authors argue that "US reinstatement during SOC could only be faithfully attributed to the respective CS1, but not to CS2, since only CS1 had been directly paired with the US, and CS2 had not previously been experienced." However, this is only strictly true for the very first trial during which the CS2 could have gained full access to the US representation.

    3. In this regard, it is unclear why the authors did not use data from the first-order conditioning phase to test for US reinstatement. Although the 4-second delay between CS1 and US is still quite short, TR-wise MVPA could provide evidence that signals are related to the CS1 and not the US itself.

    4. Relatedly, the authors perform analyses suggesting that, from early to late phases of second-order conditioning, representations of CS2 in the amygdala became more similar to US representations. Although here they attempt to model fMRI responses to the CS1 and CS2 separately, there is no evidence that this was indeed successful. As I see it, the delay between the two CS is just not long enough to dissociate these responses.

    5. Is there evidence for a CS1 evoked reinstatement of the US in the amygdala, and a CS2 evoked reinstatement of the US in the lateral OFC? In theory these signals should exist, but independently testing for activity related to the two CS requires a task design where the two CS are presented in isolation or with long enough delay between them.

  4. eLife

    Summary: All reviewers agreed that the neural mechanisms by which value is conferred to stimuli that were never directly paired with reinforcement is an important topic. However, individual reviewers raised questions regarding the study design and data analysis. In particular, reviewers agreed it was not clear how you could distinguish BOLD responses to CS1 and CS2 given the temporal proximity of their presentation. They also wondered whether the current results would provide enough advance beyond previous work.

  5. Version published to 10.1101/2020.05.28.121558 on bioRxiv