History information emerges in the cortex during learning

  1. Odeya Marmor
  2. Yael Pollak
  3. Chen Doron
  4. Fritjof Helmchen
  5. Ariel Gilad

  • Curated by eLife

    eLife logo

    eLife assessment

    This is important work analyzing the trial-by-trial progression of learning, and how the outcome of one trial influences cortex-wide neural responses on the next trial. However, the evidence for the central claims is incomplete because the potential confounds of slow hemodynamic effects and behavioral changes induced by the previous trial were not adequately addressed.

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

We learn from our experience but the underlying neuronal mechanisms incorporating past information to facilitate learning is relatively unknown. Specifically, which cortical areas encode history-related information and how is this information modulated across learning? To study the relationship between history and learning, we continuously imaged cortex-wide calcium dynamics as mice learn to use their whiskers to discriminate between two different textures. We mainly focused on comparing the same trial type with different trial history, that is, a different preceding trial. We found trial history information in barrel cortex (BC) during stimulus presentation. Importantly, trial history in BC emerged only as the mouse learned the task. Next, we also found learning-dependent trial history information in rostrolateral (RL) association cortex that emerges before stimulus presentation, preceding activity in BC. Trial history was also encoded in other cortical areas and was not related to differences in body movements. Interestingly, a binary classifier could discriminate trial history at the single trial level just as well as current information both in BC and RL. These findings suggest that past experience emerges in the cortex around the time of learning, starting from higher-order association area RL and propagating down (i.e., top-down projection) to lower-order BC where it can be integrated with incoming sensory information. This integration between the past and present may facilitate learning.

Article activity feed

  1. Version published to 10.7554/elife.83702 on eLife
  2. eLife

    Author Response

    Reviewer #1 (Public Review):

    In this manuscript, Marmor and colleagues reanalyze a previously published dataset of chronic widefield Ca2+ imaging from the dorsal cortex of mice as they learn a go/no-go somatosensory discrimination task. Comparing hit trials that have a distinct history (i.e. are preceded by distinct trial types), the authors find that hit trials preceded by correct rejections of the nontarget stimulus are associated with larger subsequent neural responses than trials precede by other hits, across the cortex. The authors analyze the time course over which this effect emerges in the barrel cortex (BC) and the rostrolateral visual area (RL), and find that its magnitude increases as the animals become expert task performers. Although the findings are potentially interesting, I, unfortunately, believe that there are important methodological concerns that could put them into question. I also disagree with the rationale that singles out BC and RL as being especially important for the emergence of trial history effects on neural responses during decision-making. I detail these points below .

    1. The authors did not perform correction for hemodynamic contamination of GCaMP fluorescence. In widefield imaging, blood vessels divisively decrease neural signals because they absorb green-wavelength photons, which could lead to crucial confounds in the interpretation of the main results because of neurovascular coupling, which lags neural activity by seconds. For example, if a reward response from the previous trial is associated with a lagged hemodynamic contamination that artificially decreases the signal in the following trial, one could get artificially higher activity in trials that were not preceded by a reward (i.e. CR), which is what the authors observed. Ideally, the experiments would be repeated with proper hemodynamic correction, but at the very least the authors should try to address this with control analyses.

    Done. We basically redone the experiment with proper hemodynamic correction and maintained trial history results. Please see point 1 above for more details (Figures S4 and S5). In addition to hemodynamic controls, we also present novel two-photon single cell data with similar results in Figure S6. We also added a dedicated section for this in the Methods section (pg. 12).

    For example, what is the time course of reward-related responses in BC and elsewhere?

    In general, and specifically in BC, reward related responses return to baseline up to 5 seconds after the start of the reward period and at least 5 seconds before the stimulus presentation of the next trial. In the novel experiments we even extended the baseline period by an additional 2 seconds just in case. Trial history information was still present with an extended inter-trial interval.

    The text now reads (pg. 4): "We further report that responses during the reward period in cortex and specifically in BC went back to baseline 4-5 seconds after the start of the reward period and 6-8 seconds before the presentation of the next stimulus (total inter-trial interval ranged between 10-12 seconds)."

    Do hemodynamics artifacts have a trial-by-trial correlation with the subsequent trial history effect?

    We have now done the proper hemodynamic control (Figure 2) and we did not find a strong effect of hemodynamic responses on trial history information.

    What is the learning time course of reward responses?

    Responses during the reward period as a function of learning were not significantly modulated. We further show the whole learning profile for BC response during the reward period in Author response image 1.

    Author response image 1.

    Response in BC averaged during the reward period (2-4 sec after texture stop) as a function of learning for each mouse separately.

    The text now reads (pg. 4): "In addition, responses in BC during the reward period were not consistently modulated as a function of learning (p>0.05; Wilcoxon signed-rank test between naïve and expert, BC response averaged during the reward period, 2-4 seconds after stimulus onset; n=7 mice). Taken together, we find that direct responses from the reward period do not effect history-related responses during the next trial."

    Note that I don't believe the FA-Hit condition analysis that the authors have already presented provides adequate control, as punishment responses are also pervasive in the cortex and therefore suffer from the same interpretational caveat. Unfortunately, I believe this is a serious methodological issue given the above. However, I will proceed to take the reported results at face value .

    We hope that our additional control analysis regarding the hemodynamic controls are satisfactory.

    1. The statistics used to assess the effect of trial history over learning are inadequate (e.g., Fig 2b). The existence of a significant effect in one condition (e.g., CR-Hit vs. Hit-Hit in expert) but not in another (e.g., same comparison in naive) does not imply that these two conditions are different. This needs to be tested directly. Moreover, the present analysis does not account for the fact that measures across learning stages are taken from the same animals. Thus, the appropriate analysis for these cases would be to first use a two-way ANOVA with repeated measures with factors of trial history and learning stage (or equivalent non-parametric test) and then derive conclusions based on post hoc pairwise tests, corrected for multiple comparisons .

    Done. We performed 2 way ANOVA as suggested and found significant history and learning effects along with a significant interaction effect for BC.

    The text now reads (pg. 4): "This difference was significant during the stim period in learning and expert phases across mice (Fig. 2b; 2-way ANOVA with repeated measures; DF (1-6) F=51 p<0.001, DF (2-12) F=18 p<0.001, DF(2-12) F=5 p<0.05 for trial history, learning and the interaction between trial history and learning; Post hoc Tukey analysis p<0.05 for trial history in learning and expert phases; p>0.05 in the naïve phase)."

    1. I am not convinced that BC and RL are especially important for trial-history-dependent effects. Figures 4 and 5 suggest that this modulation is present across the cortex, and in fact, the difference between CR-Hit and Hit-Hit in some learning stages appears stronger in other areas. BC and RL do have the highest absolute activity during the epochs in Figs 4 and 5, but I would argue that this is likely due to other aspects of the task (e.g., touch) and therefore is not necessarily relevant to the issue of trial history .

    Done. First, we would like to point out that RL during the pre period displays the largest difference between the CR-Hit and Hit-Hit conditions (Fig. 5c bottom). Second, we now show difference maps (i.e., activity in CR-Hit minus Hit-Hit) which clearly show a positive activity patch in BC during the stim period for 5 out of the 7 mice (Fig. S10a). Example maps also highlight RL during the pre period (Fig. S10b). We note that activity patches somewhat spread over to other areas and also slightly vary across mice. This is why the grand average may slightly average out trial history information. Taken together, we strongly feel that during the pre period, trial history information emerges in RL (and adjacent posterior association areas) which shift towards BC during the stim period

    Nevertheless, we agree with the reviewer that other areas (that do not necessarily display high activity) may encode trial history information and we now clearly report this in the text (pg. 5): "We note that other areas, e.g., different association areas, also encoded historydependent information especially during learning and expert phases. In addition, we present activity difference maps between CR-Hit and Hit-Hit conditions during the stim period (Fig. S10a). These maps clearly show the highest trial history information (i.e., difference in activity) in BC. Taken together, these results indicate that BC encodes history-dependent information that emerges during the stim period and just after learning. "

    And also in (pg. 6): " In addition, we present activity difference maps between CR-Hit and HitHit conditions during the pre period (Fig. S10b). These maps localize trial history information to RL which also spreads to other adjacent association areas. Moreover, activity patches slightly vary across the different mice which may affect the grand average (averaged across mice) of each area."

    1. Because of similar arguments to the above, and because this was not directly assessed, I do not believe the conclusion that history information emerges in RL and is transferred to BC is warranted. For instance, there is no direct comparison between areas, but inspection of the ROC plots in Fig 6b suggests that history information emerges concomitantly across cortical areas. I suggest directly comparing the time course between these and other areas

    Done. We now add example history AUC maps and quantify history AUC for all 25 areas during the pre and stim periods. During the pre period (Fig. 6), AUC values are concentrated around the RL (and other PPC areas), whereas during the stim periods AUC values shift to BC. Again, due to the inter-mouse variability, these differences are slightly averaged out which also makes it tough to have strong statistical test (with only 7 mice).

    The text now reads (pg. 7): "We next calculated the history AUC for each pixel during either the pre or stim period. The history AUC maps during the pre period display AUC values around the RL areas (Fig. 6f). In contrast, the history AUC maps during the stim period display AUC values mostly in BC (Fig. 6g). Quantified across 25 areas and averaged across mice, RL displays the highest history AUC during the pre period, whereas BC displays the highest history AUC values during the stim period (Fig. 6h). We note that other cortical areas such as other association areas also display high history AUC values. Taken together, we find that trial history emerges in RL before the texture arrives and then shifts to BC during stimulus presentation. "

    1. How much is task performance itself modulated by trial history? How does this change over the course of learning? These behavioral analyses would greatly help interpret the neural findings and how this trial history might be used behaviorally .

    Done, we have now calculated the dprime for Hit-Hit and CR-Hit trials separately. We find no significant differences between conditions both within and across mice (see Fig. S2 below).

    The text now reads pg. 3): "We note that learning curves that are calculated separately for each pair (i.e., either a preceding Hit or CR trial) were not significantly different (Fig. S2)."

    Reviewer #2 (Public Review):

    Marmor et al. mine a previously published dataset to examine whether recent reward/stimulus history influences responses in sensory (and other) cortices. Bulk L2/3 calcium activity is imaged across all of the dorsal cortex in transgenic mice trained to discriminate between two textures in a go/no-go behavior. The authors primarily focus on comparing responses to a specific stimulus given that the preceding trial was or was not rewarded. There are clear differences in activity during stimulus presentation in the barrel cortex along with other areas, as well as differences even before the second stimulus is presented. These differences only emerge after task learning. The data are of high quality and the paper is clear and easy to follow. My only major criticism is that I am not completely convinced that the observed difference in response is not due to differences in movement by the animal on the two trial types. That said, the demonstration of differences in sensory cortices is relatively novel, as most of the existing literature on trial history effect demonstrates such differences only in higher-order areas .

    Major :

    1a) The claim that body movements do not account for the results is in my view the greatest weakness of the paper - if the difference in response simply reflects a difference in movement, perhaps due to "excitement" in anticipation of reward after not receiving one on CR-H vs. HH trials, then this should show up in movement analysis. The authors do a little bit of this, but to me, more is needed .

    Done. We have now extensively and carefully analyzed body and whisker movements for CRHit and Hit-Hit conditions. First, In the figure below we decomposed body movements into 22 different body parts using DeepLabCut. In short, we find no significant difference between CRHit and Hit-Hit conditions in each body part separately (Fig. S7 below). This was true for the naïve, learning and expert phases. Please see additional analyses in the points below.

    This is now reported in the text (pg. 4): “In addition, we performed a more detailed body and whisker analysis, e.g., decomposing the movement to different body parts and obtaining single whisker dynamics. These analyses did not find significant differences in movement parameters between CR-Hit and Hit-Hit conditions (Fig. s7 and s8).”

    First, given the small sample size and use of non-parametric tests, you will only get p<.05 if at least 6 of the 7 mice perform in the same way. So getting p>.05 is not surprising even if there is an underlying effect. This makes it especially important to do analyses that are likely to reveal any differences; using whisker angle and overall body movement, which is poorly explained, is in my opinion insufficient. An alternative approach would be to compare movements within animals; small as the dataset is, it is feasible to do an animal-by-animal analysis, and then one could leverage the large trial count to get much greater statistical power, foregoing summary analyses that pool over only n=7 .

    We agree with this point and are have now dramatically improved our statistical analysis.

    1. We now perform within mouse statistics for responses in BC during naïve, learning and expert (see Fig. S4 below). In short, we find statistical significance for 7 out of 7 mice during the expert phase, 6 out of 7 mice in the learning phase and 0 out of 7 in the naive phase. For RL during the pre period we find significant difference in 5 out of 7 expert mice.

    This is now reported in the text (pg. 4): "In addition, a statistical comparison between CR-Hit and Hit-Hit responses within each mouse separately maintained significance for expert (7/7 mice Mann-Whitney U-test p<0.05) and learning (6/7 mice) but not for naïve (0/7 mice. Fig. S3)."

    And also in (pg. 5): "In addition, a statistical comparison between CR-Hit and Hit-Hit responses in RL within each mouse separately maintained significance for expert (5/7 mice; MannWhitney U-test p<0.05)."

    1. We would like to point out that we have now added 3 additional mice (with hemodynamics control) and performed within mouse statistics in BC and RL (Fig. S5), adding to our initial observations.

    2. In terms of body movements, we now performed within mice statistics and compared body movements between CR-Hit and Hit-Hit conditions. In general, most mice did not show a significant difference in body movements or whisker envelope.

    This is now reported in the text (pg. 4): "A within mouse statistical comparison between body or whisker parameters in CR-Hit and Hit-Hit maintained a non-significant difference in expert (1/7 mice displayed a significant difference; Mann-Whitney U-test p>0.05), learning (2/7 mice) and naïve (0/7 mice)."

    And also in (pg. 4): "Body movements and whisker parameters did not significantly differ between CR-Hit and Hit-Hit conditions during the pre-period (Similar to the stim period. Across and within mice. P>0.05; Mann-Whitney U-test)."

    In summary, we have now substantially improved our statistical analysis and further decomposed the body movements, maintaining the trial history results.

    The authors only consider a simple parametrization of movement (correlation across successive frames), and given the high variability in movement across animals, it is likely that different mice adopt different movements during the task, perhaps altering movement in specific ways. Aggregating movement across different body parts after an analysis where body parts are treated separately seems like an odd choice - perhaps it is fine, but again, supporting evidence for this is needed. As it stands, it is not clear if real differences were averaged out by combining all body parts, or what averaging actually entails .

    Please see the above point where we decomposed body movements (Fig. S7 and Methods section in Pg. 14).

    If at all possible, I would recommend examining curvature and not just the whisker angle, since the angle being the same is not too surprising given that the stimulus is in the same place. If the animal is pressing more vigorously on CR-H trials, this should result in larger curvature changes .

    Done. We now decompose whisker dynamics (i.e., curvature) using DeepLabCut (Fig. S8 see below). In general, we find no significant differences in whisker parameters between Hit-Hit and CR-Hit conditions.

    This is now reported in the text (pg. 4): "In addition, we performed a more detailed body and whisker analysis, e.g., decomposing the movement to different body parts. This analysis did not find significant differences between CR-Hit and Hit-Hit conditions (Fig. S7 and S8)."

    Finally, the authors presumably have access to lick data. Are reaction times shorter on CR-H trials? Is lick count or lick frequency shorter?

    Done. We now calculated lick reaction time and lick rate and find a significant difference for the lick reaction time but not in lick rate. We show a figure below for the reviewer and report this in the text

    The text now reads (pg. 3): "In addition, the lick reaction time (but not the lick rate) between Hit-Hit and CR-Hit were significantly different (p<0.05; Wilcoxon signed-rank test) ,maybe indicating a more considered response after a previous stop signal."

    If movement differs across trial types, it is entirely plausible that at least barrel cortex activity differences reflect differences in sensory input due to differences in whisker position/posture/etc. This would mitigate the novelty of the present results .

    As detailed above, have now meticulously analyzed the whisker parameter differences between both conditions and did not find any significant differences.

    1b) Given the importance of this control to the story, both whisker and body movement tracking frames should be explicitly shown either in the primary paper or as a supplement. Moreover, in the methods, please elaborate on how both whisker and body tracking were performed .

    Done. Please see Figs. S7 and S8 for tracking frames. This is now detailed in the above points and also the revised relevant methods section

    1. .Did streak length impact the response? For instance, in Fig. 1f "Learning", there is a 6-trial "no-go" streak; if the data are there, it would be useful to plot CR-H responses as a function of preceding unrewarded trials.

    Done. We have now calculated response in CR-Hit as a function of the number of preceding CRs. In general, we obtain inconsistent results across mice that may be due to the small number of trials that have more than one preceding CR. Nevertheless, some mice have a trend, sometimes significant, in which CR-Hit responses are higher for longer CR preceding streaks. This is especially true during the learning phase. We have decided not to include this in the manuscript and present this figure only to the reviewer.

  3. eLife

    eLife assessment

    This is important work analyzing the trial-by-trial progression of learning, and how the outcome of one trial influences cortex-wide neural responses on the next trial. However, the evidence for the central claims is incomplete because the potential confounds of slow hemodynamic effects and behavioral changes induced by the previous trial were not adequately addressed.

  4. eLife

    Reviewer #1 (Public Review):

    In this manuscript, Marmor and colleagues reanalyze a previously published dataset of chronic widefield Ca2+ imaging from the dorsal cortex of mice as they learn a go/no-go somatosensory discrimination task. Comparing hit trials that have a distinct history (i.e. are preceded by distinct trial types), the authors find that hit trials preceded by correct rejections of the non-target stimulus are associated with larger subsequent neural responses than trials precede by other hits, across the cortex. The authors analyze the time course over which this effect emerges in the barrel cortex (BC) and the rostrolateral visual area (RL), and find that its magnitude increases as the animals become expert task performers. Although the findings are potentially interesting, I, unfortunately, believe that there are important methodological concerns that could put them into question. I also disagree with the rationale that singles out BC and RL as being especially important for the emergence of trial history effects on neural responses during decision-making. I detail these points below.

    1. The authors did not perform correction for hemodynamic contamination of GCaMP fluorescence. In widefield imaging, blood vessels divisively decrease neural signals because they absorb green-wavelength photons, which could lead to crucial confounds in the interpretation of the main results because of neurovascular coupling, which lags neural activity by seconds. For example, if a reward response from the previous trial is associated with a lagged hemodynamic contamination that artificially decreases the signal in the following trial, one could get artificially higher activity in trials that were not preceded by a reward (i.e. CR), which is what the authors observed. Ideally, the experiments would be repeated with proper hemodynamic correction, but at the very least the authors should try to address this with control analyses. For example, what is the time course of reward-related responses in BC and elsewhere? Do hemodynamics artifacts have a trial-by-trial correlation with the subsequent trial history effect? What is the learning time course of reward responses? Note that I don't believe the FA-Hit condition analysis that the authors have already presented provides adequate control, as punishment responses are also pervasive in the cortex and therefore suffer from the same interpretational caveat. Unfortunately, I believe this is a serious methodological issue given the above. However, I will proceed to take the reported results at face value.

    2. The statistics used to assess the effect of trial history over learning are inadequate (e.g., Fig 2b). The existence of a significant effect in one condition (e.g., CR-Hit vs. Hit-Hit in expert) but not in another (e.g., same comparison in naive) does not imply that these two conditions are different. This needs to be tested directly. Moreover, the present analysis does not account for the fact that measures across learning stages are taken from the same animals. Thus, the appropriate analysis for these cases would be to first use a two-way ANOVA with repeated measures with factors of trial history and learning stage (or equivalent non-parametric test) and then derive conclusions based on post hoc pairwise tests, corrected for multiple comparisons.

    3. I am not convinced that BC and RL are especially important for trial-history-dependent effects. Figures 4 and 5 suggest that this modulation is present across the cortex, and in fact, the difference between CR-Hit and Hit-Hit in some learning stages appears stronger in other areas. BC and RL do have the highest absolute activity during the epochs in Figs 4 and 5, but I would argue that this is likely due to other aspects of the task (e.g., touch) and therefore is not necessarily relevant to the issue of trial history.

    4. Because of similar arguments to the above, and because this was not directly assessed, I do not believe the conclusion that history information emerges in RL and is transferred to BC is warranted. For instance, there is no direct comparison between areas, but inspection of the ROC plots in Fig 6b suggests that history information emerges concomitantly across cortical areas. I suggest directly comparing the time course between these (and other areas).

    5. How much is task performance itself modulated by trial history? How does this change over the course of learning? These behavioral analyses would greatly help interpret the neural findings and how this trial history might be used behaviorally.

  5. eLife

    Reviewer #2 (Public Review):

    Marmor et al. mine a previously published dataset to examine whether recent reward/stimulus history influences responses in sensory (and other) cortices. Bulk L2/3 calcium activity is imaged across all of the dorsal cortex in transgenic mice trained to discriminate between two textures in a go/no-go behavior. The authors primarily focus on comparing responses to a specific stimulus given that the preceding trial was or was not rewarded. There are clear differences in activity during stimulus presentation in the barrel cortex along with other areas, as well as differences even before the second stimulus is presented. These differences only emerge after task learning. The data are of high quality and the paper is clear and easy to follow. My only major criticism is that I am not completely convinced that the observed difference in response is not due to differences in movement by the animal on the two trial types. That said, the demonstration of differences in sensory cortices is relatively novel, as most of the existing literature on trial history effect demonstrates such differences only in higher-order areas.

    Major:

    1a. The claim that body movements do not account for the results is in my view the greatest weakness of the paper - if the difference in response simply reflects a difference in movement, perhaps due to "excitement" in anticipation of reward after not receiving one on CR-H vs. H-H trials, then this should show up in movement analysis. The authors do a little bit of this, but to me, more is needed.

    First, given the small sample size and use of non-parametric tests, you will only get p<.05 if at least 6 of the 7 mice perform in the same way. So getting p>.05 is not surprising even if there is an underlying effect. This makes it especially important to do analyses that are likely to reveal any differences; using whisker angle and overall body movement, which is poorly explained, is in my opinion insufficient. An alternative approach would be to compare movements within animals; small as the dataset is, it is feasible to do an animal-by-animal analysis, and then one could leverage the large trial count to get much greater statistical power, foregoing summary analyses that pool over only n=7.

    The authors only consider a simple parametrization of movement (correlation across successive frames), and given the high variability in movement across animals, it is likely that different mice adopt different movements during the task, perhaps altering movement in specific ways. Aggregating movement across different body parts after an analysis where body parts are treated separately seems like an odd choice - perhaps it is fine, but again, supporting evidence for this is needed. As it stands, it is not clear if real differences were averaged out by combining all body parts, or what averaging actually entails.

    If at all possible, I would recommend examining curvature and not just the whisker angle, since the angle being the same is not too surprising given that the stimulus is in the same place. If the animal is pressing more vigorously on CR-H trials, this should result in larger curvature changes.

    Finally, the authors presumably have access to lick data. Are reaction times shorter on CR-H trials? Is lick count or lick frequency shorter?

    If movement differs across trial types, it is entirely plausible that at least barrel cortex activity differences reflect differences in sensory input due to differences in whisker position/posture/etc. This would mitigate the novelty of the present results.

    1b. Given the importance of this control to the story, both whisker and body movement tracking frames should be explicitly shown either in the primary paper or as a supplement. Moreover, in the methods, please elaborate on how both whisker and body tracking were performed.

    2. Did streak length impact the response? For instance, in Fig. 1f "Learning", there is a 6-trial "no-go" streak; if the data are there, it would be useful to plot CR-H responses as a function of preceding unrewarded trials.

  6. Version published to 10.1101/2022.11.01.514667v1 on bioRxiv