Activity Subspaces in Medial Prefrontal Cortex Distinguish States of the World

This article has been Reviewed by the following groups

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Abstract

Medial prefrontal cortex (mPfC) activity represents information about the state of the world, including present behavior, such as decisions, and the immediate past, such as short-term memory. Unknown is whether information about different states of the world are represented in the same mPfC neural population and, if so, how they are kept distinct. To address this, we analyze here mPfC population activity of male rats learning rules in a Y-maze, with self-initiated choice trials to an arm end followed by a self-paced return during the intertrial interval (ITI). We find that trial and ITI population activity from the same population fall into different low-dimensional subspaces. These subspaces encode different states of the world: multiple features of the task can be decoded from both trial and ITI activity, but the decoding axes for the same feature are roughly orthogonal between the two task phases, and the decodings are predominantly of features of the present during the trial but features of the preceding trial during the ITI. These subspace distinctions are carried forward into sleep, where population activity is preferentially reactivated in post-training sleep but differently for activity from the trial and ITI subspaces. Our results suggest that the problem of interference when representing different states of the world is solved in mPfC by population activity occupying different subspaces for the world states, which can be independently decoded by downstream targets and independently addressed by upstream inputs.

SIGNIFICANCE STATEMENT Activity in the medial prefrontal cortex plays a role in representing the current and past states of the world. We show that during a maze task, the activity of a single population in medial prefrontal cortex represents at least two different states of the world. These representations were sequential and sufficiently distinct that a downstream population could separately read out either state from that activity. Moreover, the activity representing different states is differently reactivated in sleep. Different world states can thus be represented in the same medial prefrontal cortex population but in such a way that prevents potentially catastrophic interference between them.

Article activity feed

  1. ##Author Response

    We thank the editors and reviewers for taking the time to assess our paper. We note that the reviewers seemed generally supportive of the paper, including noting that the paper addressed important questions. For context, we reiterate here our main findings:

    • a prefrontal cortex population encodes the past and the present in its joint activity, but solves the interference problem by encoding all features on independent axes for their past and their present.
    • This encoding would in principle allow upstream regions to independently access representations of the past and present in mPfC populations. We go on to show this happens: we show that only the encoding of the present, and not the past, is reactivated in sleep after training.

    In this context, the main editorial objection that we “did not control for potential confounding of behavioral variables” is not explained in the reviews; we also note that there were no “concerns about the analytical methods used” that were pertinent to our main findings. We are thus unclear about the basis for rejection.

    We respond below to the main points of each reviewer; their suggestions on terminology and of separating literature citations on rodent and primate PfC are being given due consideration.

    ###Reviewer #1:

    Maggi and Humphries examined how the coding of the present and past choices in the medial prefrontal cortex (mPFC) of the rats during a Y-maze task overlaps and whether they can be reliably distinguished. They found that the neural signals related to the animal's choice in the present and past are distinct and as a result they can be recalled separately, for example, during post-training sleep. Although these are very important questions and an interesting set of analyses have been applied, the results in this report are not entirely convincing, because the analyses did not successfully exclude some alternative hypotheses.

    1. The authors analyzed the signals related to the choice, light cue, and outcome separately, and this is possible because the relationship between the animal's choices and cues were decoupled by testing the animals under at least two different rules. There were a total of 4 alternative rules and different sessions included different subsets of these rules. It is possible that at least some results reported in this paper might vary depending on which of these results were tested. For example, rules might affect how the animals learned the task. Therefore, the authors should provide more detailed information about how often different rules were used to collect the neural data reported in this paper, and whether any of the results change according to the rules used in a given session.

    In the paper we did examine mPfC encoding in the trials under the two qualitatively distinct types of rule (direction-based i.e. egocentric, and cue-based i.e. allocentric), and showed that encoding of the direction, light, and outcome occurred in both rule types (figure 1e). We gave the number of sessions for those rules in the legend for Figure 1e. (We could equally decode all 3 features in direction-based and cue-based rule sessions in the inter-trial interval as well, see Maggi et al 2018, Figure 9). Thus we compared the decoding vectors across all rule-types.

    Only 8 sessions contained more than 1 rule, in the sessions in which the rule was switched. In the full analysis underlying this paper, we had also separately examined the decoding in these 8 rule-switch sessions, and found equally good decoding of direction, choice, and cue. As the paper was already dense - see e.g. Reviewer 3’s comments - we elected to not show this null result in the current version of the manuscript - it is available in version 1 of this preprint - but it can be restored if desired.

    1. The authors claim that the neural coding identified in this study does not depend on the signals in individual neurons by showing comparable results after removing the neurons with significant modulations. This logic is flawed, because the neurons without "significant" modulations might still include meaningful signals due to type II errors. Furthermore, if individual neurons carry absolutely no signals, how can a population of neurons still encode any signals? This might suggest some kind of joint coding, and the authors should not merely implicate such a possibility without more thorough tests.

    The joint coding of information by a population of neurons is the basis for the whole paper, and is tested extensively: for example, Figure 1 is about establishing that joint coding exists in mPfC. Our point on lines 91-95 was simply to show that the decoding could not be trivially explained by one or two neurons that reliably and strongly differed in the firing rates between different labels (e.g. between left or right choice of direction). To do so, we found sessions in which there were neurons with significantly detectable tuning to the task feature, omitted those sessions, and then looked at the performance of the feature decoding in the remaining sessions - and found it was just as good. Indeed, our point is precisely that it is possible for individual neurons to carry no signals detectable by classic significance testing (potentially due to Type II errors), yet for the population to be able to perfectly encode the information.

    The explanation is simply that most, and sometimes all, individual neurons do not consistently covary their firing with the changes in a feature (e.g. choose left and choose right trials) across every trial of a session. In other words, no neuron need consistently participate in encoding information. But so long as when a neuron does change its firing it does consistently vary with the feature, then across a population there are enough intermittently participating neurons on a given trial to always decode the information.

    1. The authors analyzed the activity divided into 5 different epochs, where the position #3 corresponds to a choice point and #5 corresponds to the reward site. Therefore, it is surprising that the reliable outcome signals begin to emerge from the position #3 (i.e., choice point). Is this a false positive?

    No, this replicated a common finding of outcome-predictive signals in prefrontal cortex; e.g. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Fellows, L. K. Advances in understanding ventromedial prefrontal function: the accountant joins the executive. Neurology 68, 991–995 (2007).

    Sul, J. H., Kim, H., Huh, N., Lee, D. & Jung, M. W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449–460 (2010).

    Kaplan, R. et al. The neural representation of prospective choice during spatial planning and decisions. PLoS Biol. 15, e1002588 (2017).

    We will add these references to the next version of the manuscript.

    1. The authors report that there is retrospective coding, i.e., no coding of the choice in the previous. By contrast, during the intertrial interval (while the animal's returning to the start position), the signals related to the "past" choice were still present but different from how this information was coding earlier during the trial. This is not surprising since during the intertrial interval, the animal's movement direction is opposite compared to that during the trial, so this coding change could reflect the animal's sensory environment. Whether the brain encodes the past and previous events using different coding schemes or not cannot be tested with such confounding.

    We note that the reviewer’s objection here only relates to the choice of arm direction, whereas we showed independent encoding of all three features: direction, outcome, and cue position. We can thus test how the past and present are differently encoded because we showed they are both encoded in the same set of neurons. We showed at length both here (Figure 2a&c, Supplementary Figure 5a) and in Maggi et al 2018 (Figs 5-6 and accompanying supplementary figures) that we could decode the past events from the population activity during the inter-trial interval. The information of the trial and the inter-trial interval can be decoded from the same neurons, so the question is: how can the same neurons encode both the present and the past?

    One interpretation of the reviewer’s comments is that they are concerned about the possible confounding of movement direction between the trial and the following inter-trial interval. Namely, that the turn directions are guaranteed to be opposite: e.g a left turn into the left-hand arm on the trial would mean a right-hand turn on the return journey of the inter-trial interval. However, that would mean the feature labels would be exactly complementary e.g. trial =[L L R L R] and ITI = [R R L R L]. So if the population was encoding the direction choice the same way in both the trial and ITI, then using the trial’s decoder of direction to decode direction choice in the ITI should result in a performance of 1-[proportion of correctly classified trials], meaning the classifier would be significantly below chance (and vice-versa for using the inter-trial interval’s decoder for the trials). However, we find the cross-decoding performs at chance (Fig 2).

    1. The authors tested whether the coding of present and past events is consistent using a transfer (cross-decoding) analysis. However, this is based on simply correlation, and does not exclude the possibility that neurons changing their activity similarly according to (for example) the animal's choice might also change their baseline activity between the two periods (as revealed by the analysis of "population activity" in Figure 3) or might additionally encode different variables. In this case, decoding based on simple correlation might not reveal consistent coding that might be present.

    It is unclear what the referee means by the cross-decoding analysis being “based on simple correlation”. The decoder is trained on vectors of firing rates (cf Figure 1b). The decoder assigns high weights to neurons whose activity differs most strongly between the two labels (e.g. left and right choice of direction). So a change in “baseline”, presumably meaning the average firing rate of a neuron across all trials or all ITIs, would not alter the decoder outcome. In addition to the two cross-decoding tests, we also showed the independent encoding by: (a) The angles formed by the decoding vectors trained solely on the trials and solely on the ITIs (Fig 2d-f) (b) The independence of the population rate vectors between trials and ITIs (Fig 3). Indeed, the change in population rates between trials and ITIs shown in Figure 3 is exactly those predicted by the cross-decoding results, as explained on pg 7.

    ###Reviewer #2:

    The study by Maggi and Humphries re-examines data by Peyrache et al. (2009), which the authors have themselves analysed previously (Maggi et al., 2018), recorded , in rat prelimbic/infralimbic cortex (see comment below on terminology). In particular, they look at the relationship between decoding of task events during performance of a trial, and during the subsequent intertrial interval. (n.b. in this study, unlike in many studies, the ITI is considerably longer than the trial period). They find that although task-relevant information can be decoded during these two periods, the information is encoded in orthogonal subspaces during trials ('the present') and ITIs ('the past'). They build on this to examine how information is encoded during sleep following training (vs a pre-training control period). They find that only the trial subspaces are reactivated during sleep, not the ITI subspaces, and more so if the rat received a higher rate of average reward.

    On the whole, I found this an interesting paper with a clear set of findings, and well-analysed data. Although the advance in some ways an incremental one on previous studies of sleep/replay, and on the authors' previous analyses of this dataset, the study will undoubtedly be of interest to researchers who are interested in consolidation of past experience during sleep. In particular, the study benefits from being able to look for two different types of information ('past' and 'present' decoders) in the same sleep recording sessions. There were a few things that I felt the authors could address:

    1. For the cross-decoding analysis in figure 2 b, it is not entirely clear from the main text which part of the trial and ITI coding is being used here. It seems to me like a more useful way of showing the cross-decoding analysis would be to show the 10x10 matrix of cross decoding accuracy for each of the 5 maze positions in both trials and ITIs. This is, I think, different from what the analysis in figure 3g is trying to show (which plots the classification error after dimensionality reduction to a 2D space).

    As we strived to explain in the text, for the cross-decoding analysis we used the decoder trained on the firing rates across the entire trial and separately across the entire ITI, in order to arrive at the most stable decoding vectors. We did not show the cross-decoding for the full 5x5 matrix of positions, as the results would be quite noisy. Nevertheless, this is a constructive suggestion, and we will add this analysis. (And indeed the analysis in Figure 3 already shows that the population activity is separable in 1 or 2 dimensions between the trials and ITIs at each maze position, so we would expect the decoder weight vectors to also be independent).

    1. It was surprising to me that the authors do not mention the finding in figure 4e anywhere in the abstract or introduction. It makes the reactivation story far more compelling if it can be linked to a change in behaviour during the preceding trials. I think this finding would benefit from not being buried deep in the results section.

    We are happy to make this result clearer. Our main finding is of the independent coding, and this result in Fig 4e does not speak directly to the independent coding results, but rather is a lovely little result to support the hypothesis that there really is reactivation of the population vectors in sleep. Because it did not speak to the main thrust of the paper, it was omitted from the abstract given the constraints on the number of words (150).

    1. The finding in figure 5 seems slightly extra-ordinary. It suggests that reactivation decoding during sleep is reliable even if very long bins of activity are used to calculate the firing rate (e.g. up to 10s). Does this relationship ever break down? Presumably with the sleep data, it would be possible to extend bins up to 1 minute, 5 minutes, etc. If there is still more reactivation at these extremely long time-bin lengths, does this mean that these neurons are essentially more persistently active? One possible way to test for this might be to project the data recorded during sleep through the classifier weights, and then calculate the autocorrelation function of this projected data (e.g. Murray et al., Nat Neuro 2014) - if this activity becomes more persistent, the shape of the ACF may change post-training.

    An excellent question. Rather than persistent activity, we interpreted the consistency of reactivation across orders of magnitude time-scales as showing that the correlations between the neurons were roughly consistent; and thus when active tended to be active in roughly the same relative order. Support for this comes from the findings in Appendix Fig A4e - the correlation matrix between neurons in the trial was more consistently found in post than pre-session sleep.

    ###Reviewer #3:

    This article asks the question if within trial (present) and ITI (past) task parameters are encoded in mPFC, and how encoding during these two trial epochs are encoded. They claim that firing in mPFC reflects past and present, but population encoding of past and present are independent. Further they show that the present is reactivated during sleep, not the past.

    On the face of it, this seems like an interesting paper. It is novel in that ITI encoding would be highly related to what was going on in the trial. The sleep finding is also interesting but I don't quite get the distinction between present and past for sleep. That could use some clarification.

    1. I'm not an expert in regards to this type of analysis, but throughout I was left with the feeling that I would prefer at least some single neuron data and firing rate analysis to complement the highly computational analysis, which frankly, was difficult to understand or critique by somebody who is not an expert.

    The goal of the paper is to assess the population coding in PfC of the same events in the past and the present. Indeed, as reported in the paper, we found 25-39 sessions which had no single neuron tuning at all to a given event in a trial (such as the choice of maze arm).

    1. I would have liked to see more analysis of firing correlations with behavior. It seems to me if animals were doing different things during the trial and the ITI, then it might not be a surprise that there is independent encoding.
    1. I also wonder if the finding is solely dependent on the task (which is poorly described). It seems like there should be independent coding of past and present in this circumstance because they do not feed into each other, and behavior during one is independent of behavior in the other.
    1. Relatedly, the authors suggest that independent encoding can explain how the brain resolves interference between past and present, but in this task there was no interference between past and present, and the authors do not show that when there is more or less dependent encoding that there is more or less interference. Without it is unclear how to know how important this finding is as it relates to performance and general mPFC function.

    We deal with these points together, as they are all on the behaviour in the trial and inter-trial interval in the task. Yes, the behaviour in the trial is independent of that in the inter-trial interval, so there is no “interference” of behaviour. But that is not of relevance to what is encoded in the PfC. The Introduction and Discussion both point out that the problem is interference of the encoding itself: the encoding of the past and present exists, as we show at length, so the question is: how can it co-exist in the same neurons? We indeed ask if there is no “interference” in the encoding simply because activity in the inter-trial interval is just a memory trace of activity in the trial, and rule that out.

    We cannot address when there is “more or less dependent” encoding, because the results are what they are: there is independent encoding of the same events (Figure 2).

    The task is described in detail in the Methods (pgs 20-21).

    1. Could activity reflect what the animal predicts will happen on the next trial, or what they are planning to do? It wasn't clear if that was examined.

    Whether activity in the inter-trial interval predicted what will happen in the next trial was examined in detail in Maggi et al 2018 (Fig 6), and shown here in Figure 2g. We found no encoding of the following trial’s choices, except for a very niche occurrence: an above chance decoding of the next trial’s direction choice when the rat had returned to the start position, during a learning session, and for a direction rule. In other words, as it turned to start the next trial, so there was decoding of the upcoming choice of arm.

  2. ###Reviewer #3:

    This article asks the question if within trial (present) and ITI (past) task parameters are encoded in mPFC, and how encoding during these two trial epochs are encoded. They claim that firing in mPFC reflects past and present, but population encoding of past and present are independent. Further they show that the present is reactivated during sleep, not the past.

    On the face of it, this seems like an interesting paper. It is novel in that ITI encoding would be highly related to what was going on in the trial. The sleep finding is also interesting but I don't quite get the distinction between present and past for sleep. That could use some clarification.

    1. I'm not an expert in regards to this type of analysis, but throughout I was left with the feeling that I would prefer at least some single neuron data and firing rate analysis to complement the highly computational analysis, which frankly, was difficult to understand or critique by somebody who is not an expert.

    2. I would have liked to see more analysis of firing correlations with behavior. It seems to me if animals were doing different things during the trial and the ITI, then it might not be a surprise that there is independent encoding.

    3. I also wonder if the finding is solely dependent on the task (which is poorly described). It seems like there should be independent coding of past and present in this circumstance because they do not feed into each other, and behavior during one is independent of behavior in the other.

    4. Relatedly, the authors suggest that independent encoding can explain how the brain resolves interference between past and present, but in this task there was no interference between past and present, and the authors do not show that when there is more or less dependent encoding that there is more or less interference. Without it is unclear how to know how important this finding is as it relates to performance and general mPFC function.

    5. Could activity reflect what the animal predicts will happen on the next trial, or what they are planning to do? It wasn't clear if that was examined.

    6. I have some issue with the definition of past and present in the context of this task. More justification should be provided.

  3. ###Reviewer #2:

    The study by Maggi and Humphries re-examines data by Peyrache et al. (2009), which the authors have themselves analysed previously (Maggi et al., 2018), recorded , in rat prelimbic/infralimbic cortex (see comment below on terminology). In particular, they look at the relationship between decoding of task events during performance of a trial, and during the subsequent intertrial interval. (n.b. in this study, unlike in many studies, the ITI is considerably longer than the trial period). They find that although task-relevant information can be decoded during these two periods, the information is encoded in orthogonal subspaces during trials ('the present') and ITIs ('the past'). They build on this to examine how information is encoded during sleep following training (vs a pre-training control period). They find that only the trial subspaces are reactivated during sleep, not the ITI subspaces, and more so if the rat received a higher rate of average reward.

    On the whole, I found this an interesting paper with a clear set of findings, and well-analysed data. Although the advance in some ways an incremental one on previous studies of sleep/replay, and on the authors' previous analyses of this dataset, the study will undoubtedly be of interest to researchers who are interested in consolidation of past experience during sleep. In particular, the study benefits from being able to look for two different types of information ('past' and 'present' decoders) in the same sleep recording sessions. There were a few things that I felt the authors could address:

    1. For the cross-decoding analysis in figure 2 b, it is not entirely clear from the main text which part of the trial and ITI coding is being used here. It seems to me like a more useful way of showing the cross-decoding analysis would be to show the 10x10 matrix of cross decoding accuracy for each of the 5 maze positions in both trials and ITIs. This is, I think, different from what the analysis in figure 3g is trying to show (which plots the classification error after dimensionality reduction to a 2D space).

    2. It was surprising to me that the authors do not mention the finding in figure 4e anywhere in the abstract or introduction. It makes the reactivation story far more compelling if it can be linked to a change in behaviour during the preceding trials. I think this finding would benefit from not being buried deep in the results section.

    3. The finding in figure 5 seems slightly extra-ordinary. It suggests that reactivation decoding during sleep is reliable even if very long bins of activity are used to calculate the firing rate (e.g. up to 10s). Does this relationship ever break down? Presumably with the sleep data, it would be possible to extend bins up to 1 minute, 5 minutes, etc. If there is still more reactivation at these extremely long time-bin lengths, does this mean that these neurons are essentially more persistently active? One possible way to test for this might be to project the data recorded during sleep through the classifier weights, and then calculate the autocorrelation function of this projected data (e.g. Murray et al., Nat Neuro 2014) - if this activity becomes more persistent, the shape of the ACF may change post-training.

    4. I disagree with the use of the term 'medial prefrontal cortex' to describe this area of the rodent brain. Although this is the term used in the original paper by Battaglia et al. (2009), I would suggest the authors use the more anatomically precise description of 'prelimbic/infralimbic cortex', and mention that the recordings are ~2.7mm anterior to bregma (see supplementary figure 1 of Battaglia 2009 paper; see Laubach et al., eNeuro 2018 for further discussion on terminology). Also, when the authors discuss these recordings in the context of the wider literature, it is difficult to know how to relate activity in this dysgranular region of the rodent brain to regions of granular prefrontal cortex in the primate brain - given the anatomical correspondence between rodents and primates is very uncertain for these granular regions (e.g. citations to Schuck et al., 2015; Averbeck and Lee, 2006; etc). It would be good to acknowledge this somewhere.

  4. ###Reviewer #1:

    Maggi and Humphries examined how the coding of the present and past choices in the medial prefrontal cortex (mPFC) of the rats during a Y-maze task overlaps and whether they can be reliably distinguished. They found that the neural signals related to the animal's choice in the present and past are distinct and as a result they can be recalled separately, for example, during post-training sleep. Although these are very important questions and an interesting set of analyses have been applied, the results in this report are not entirely convincing, because the analyses did not successfully exclude some alternative hypotheses.

    1. The authors analyzed the signals related to the choice, light cue, and outcome separately, and this is possible because the relationship between the animal's choices and cues were decoupled by testing the animals under at least two different rules. There were a total of 4 alternative rules and different sessions included different subsets of these rules. It is possible that at least some results reported in this paper might vary depending on which of these results were tested. For example, rules might affect how the animals learned the task. Therefore, the authors should provide more detailed information about how often different rules were used to collect the neural data reported in this paper, and whether any of the results change according to the rules used in a given session.

    2. The authors claim that the neural coding identified in this study does not depend on the signals in individual neurons by showing comparable results after removing the neurons with significant modulations. This logic is flawed, because the neurons without "significant" modulations might still include meaningful signals due to type II errors. Furthermore, if individual neurons carry absolutely no signals, how can a population of neurons still encode any signals? This might suggest some kind of joint coding, and the authors should not merely implicate such a possibility without more thorough tests.

    3. The authors analyzed the activity divided into 5 different epochs, where the position #3 corresponds to a choice point and #5 corresponds to the reward site. Therefore, it is surprising that the reliable outcome signals begin to emerge from the position #3 (i.e., choice point). Is this a false positive?

    4. The authors report that there is retrospective coding, i.e., no coding of the choice in the previous. By contrast, during the intertrial interval (while the animal's returning to the start position), the signals related to the "past" choice were still present but different from how this information was coding earlier during the trial. This is not surprising since during the intertrial interval, the animal's movement direction is opposite compared to that during the trial, so this coding change could reflect the animal's sensory environment. Whether the brain encodes the past and previous events using different coding schemes or not cannot be tested with such confounding.

    5. The authors tested whether the coding of present and past events is consistent using a transfer (cross-decoding) analysis. However, this is based on simply correlation, and does not exclude the possibility that neurons changing their activity similarly according to (for example) the animal's choice might also change their baseline activity between the two periods (as revealed by the analysis of "population activity" in Figure 3) or might additionally encode different variables. In this case, decoding based on simple correlation might not reveal consistent coding that might be present.

    6. Given the length of the inter-trial interval, it might be informative to examine whether neurons activity during the early part of the inter-trial interval might get reactively differently during sleep compared to those becoming active later during the intertrial interval.

  5. ##Preprint Review

    This preprint was reviewed using eLife’s Preprint Review service, which provides public peer reviews of manuscripts posted on bioRxiv for the benefit of the authors, readers, potential readers, and others interested in our assessment of the work. This review applies only to version 2 of the manuscript. Daeyeol Lee (Johns Hopkins University) served as the Reviewing Editor.

    ###Summary:

    Although the reviewers have acknowledged the significance of better understanding how neurons in the prefrontal cortex can simultaneously encode signals related to the animal's present and past behaviors, they were concerned that the findings reported in this paper did not control for potential confounding of behavioral variables during the epochs analyzed in this manuscript. They also raised several concerns about the analytical methods used. In the consultation, the reviewers all agree that the advance represented here is not to the level that would be expected by readers and the overall enthusiasm was limited.