Coding of latent variables in sensory, parietal, and frontal cortices during closed-loop virtual navigation
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
The manuscript by Noel et al reports parallel neurophysiological responses from the three brain areas MST, 7a and dlPFC of monkeys during a novel behavioural paradigm developed by the same group previously. The continual nature of this paradigm with a closed action-perception loop makes the animal behaviour more naturalistic compared to classical paradigms with artificial breaks between sensory stimulation and action. Findings of neurophysiology under such a paradigm are novel and of broad interest to cognitive and systems neuroscientists. The data presented in the paper support the claim of distributed neural coding in which task-specific sub-networks may form.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #2 agreed to share their name with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
We do not understand how neural nodes operate and coordinate within the recurrent action-perception loops that characterize naturalistic self-environment interactions. Here, we record single-unit spiking activity and local field potentials (LFPs) simultaneously from the dorsomedial superior temporal area (MSTd), parietal area 7a, and dorsolateral prefrontal cortex (dlPFC) as monkeys navigate in virtual reality to ‘catch fireflies’. This task requires animals to actively sample from a closed-loop virtual environment while concurrently computing continuous latent variables: (i) the distance and angle travelled (i.e., path integration) and (ii) the distance and angle to a memorized firefly location (i.e., a hidden spatial goal). We observed a patterned mixed selectivity, with the prefrontal cortex most prominently coding for latent variables, parietal cortex coding for sensorimotor variables, and MSTd most often coding for eye movements. However, even the traditionally considered sensory area (i.e., MSTd) tracked latent variables, demonstrating path integration and vector coding of hidden spatial goals. Further, global encoding profiles and unit-to-unit coupling (i.e., noise correlations) suggested a functional subnetwork composed by MSTd and dlPFC, and not between these and 7a, as anatomy would suggest. We show that the greater the unit-to-unit coupling between MSTd and dlPFC, the more the animals’ gaze position was indicative of the ongoing location of the hidden spatial goal. We suggest this MSTd-dlPFC subnetwork reflects the monkeys’ natural and adaptive task strategy wherein they continuously gaze toward the location of the (invisible) target. Together, these results highlight the distributed nature of neural coding during closed action-perception loops and suggest that fine-grain functional subnetworks may be dynamically established to subserve (embodied) task strategies.
Article activity feed
-
-
-
Evaluation Summary:
The manuscript by Noel et al reports parallel neurophysiological responses from the three brain areas MST, 7a and dlPFC of monkeys during a novel behavioural paradigm developed by the same group previously. The continual nature of this paradigm with a closed action-perception loop makes the animal behaviour more naturalistic compared to classical paradigms with artificial breaks between sensory stimulation and action. Findings of neurophysiology under such a paradigm are novel and of broad interest to cognitive and systems neuroscientists. The data presented in the paper support the claim of distributed neural coding in which task-specific sub-networks may form.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested …
Evaluation Summary:
The manuscript by Noel et al reports parallel neurophysiological responses from the three brain areas MST, 7a and dlPFC of monkeys during a novel behavioural paradigm developed by the same group previously. The continual nature of this paradigm with a closed action-perception loop makes the animal behaviour more naturalistic compared to classical paradigms with artificial breaks between sensory stimulation and action. Findings of neurophysiology under such a paradigm are novel and of broad interest to cognitive and systems neuroscientists. The data presented in the paper support the claim of distributed neural coding in which task-specific sub-networks may form.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #2 agreed to share their name with the authors.)
-
Reviewer #1 (Public Review):
In their paper, Noel, Angelaki and colleagues investigate neural coding in an innovative closed-loop sensorimotor task, where monkeys navigate to a "firefly" target with a joystick in a virtual reality set-up. They collect an impressive data set of hundreds of single neurons from areas MST, 7a and dlPFC. They analyse the data set by fitting spike trains to a Poisson Generalized Additive Model (P-GAM) to discern the different influences (e.g. task variables, hidden variables) have on firing rates.
The strengths of the manuscript lie in the innovative task that relies closed-loop perception-action integration, the large data-set of single cells across sensory, parietal and frontal cortices and the novel analysis approach to this complex data set.
Weaknesses lie in the complexity of the data set and analyses …
Reviewer #1 (Public Review):
In their paper, Noel, Angelaki and colleagues investigate neural coding in an innovative closed-loop sensorimotor task, where monkeys navigate to a "firefly" target with a joystick in a virtual reality set-up. They collect an impressive data set of hundreds of single neurons from areas MST, 7a and dlPFC. They analyse the data set by fitting spike trains to a Poisson Generalized Additive Model (P-GAM) to discern the different influences (e.g. task variables, hidden variables) have on firing rates.
The strengths of the manuscript lie in the innovative task that relies closed-loop perception-action integration, the large data-set of single cells across sensory, parietal and frontal cortices and the novel analysis approach to this complex data set.
Weaknesses lie in the complexity of the data set and analyses that make it difficult for the reader to relate the results back to the literature of single units intensively characterised with optimised stimuli and more traditional tasks. This would allow the reader to potentially distinguish neural coding that is central to the particular task performance from unrelated signals and fully assess the novelty of the results. Further information on strength of unit tuning, responsiveness, task lateralisation, visual stimulus patterns and other methodological information would be helpful.
This work is of potentially considerable impact on the field as it is trying to capture the dynamic of neural coding across many single neurons in a closed-loop sensori-motor task.
-
Reviewer #2 (Public Review):
Even though in real life, action and perception are intimately linked in a closed loop system, many neuroscience experiments completely separate them in order to study cognitive processes in isolation. The current manuscript addresses this important shortcoming in a goal-directed virtual navigation task in which monkeys catch memorized 'fireflies'. Simultaneous recordings from three brain areas show that the many components of this task are represented in the brain in a distributed manner. This paper is important as it studies the brain under natural circumstances, highlights the distributed nature of neural coding under such circumstances, and demonstrates how new analysis methods that don't rely on cross-trial averaging can be used. As such, it is of interest to a wide range of cognitive and systems …
Reviewer #2 (Public Review):
Even though in real life, action and perception are intimately linked in a closed loop system, many neuroscience experiments completely separate them in order to study cognitive processes in isolation. The current manuscript addresses this important shortcoming in a goal-directed virtual navigation task in which monkeys catch memorized 'fireflies'. Simultaneous recordings from three brain areas show that the many components of this task are represented in the brain in a distributed manner. This paper is important as it studies the brain under natural circumstances, highlights the distributed nature of neural coding under such circumstances, and demonstrates how new analysis methods that don't rely on cross-trial averaging can be used. As such, it is of interest to a wide range of cognitive and systems neuroscientists.
Strengths
- The first main strength of this paper lies in the experimental design. Catching fireflies in a virtual reality environment engages the brain in a much more naturalistic way than any 'traditional' goal-directed behaviour task used in monkeys. Therefore, also the neural activity that is recorded is much more likely to resemble the processes happening naturally in the brain.
- The other main strength lies in the analysis methods used. The 'firefly task' does not result in identical trials that can be averaged post-hoc to distill the average brain response of a certain area. Instead, a P-GAM model is fit to single-neuron activity which can take an unlimited number of variables (e.g. task-related, behavioural state-related, brain dynamics) as input and computes, for each neuron, the combination of these variables that best fit the data. This approach is innovative and can be used in a much broader range of data and experimental designs than the one described here.Weaknesses
- The number of recorded neurons in the three areas differs greatly: 231 units in MSTd, 823 units in dlPFC, and 3200 units in area 7a. Yet many conclusions in the paper rely on neuronal numbers: the fractions of neurons tuned to certain sensorimotor and latent variables differ between the areas, the variables explaining the firing rates cluster differently in the neurons of the three areas, and both the coarse LFP connectivity and the fine unit-to-unit coupling within areas differ. Especially the clustering results might depend on the number of recorded neurons: the fact that almost all MSTd and dlPFC neurons are categorized as belonging to the same cluster, whereas the area 7a neurons appear in three distinct clusters, could be caused by the much larger number of recorded neurons in area 7a. Also unit-to-unit coupling is more likely to show up in the data with a much larger number of recorded neurons. The data could be corrected for these differences in number of recorded neurons.
- The P-GAM model is a great analysis tool for these kinds of data. However, the variables that the authors put into it are conceptually very different from each other. There are purely external task variables such as target onset and offset, latent variables such as distance to target that require knowledge of one's own position in space, and purely internal brain dynamics variables such as coupling to the LFP in another area. In that light, the finding of 'many variables contributing to the responses' is not surprising; all neurons in the brain are probably influenced both by external variables and internal brain dynamics. Maybe the authors could comment on the different nature of their variables and how that impacts their results.
- The authors claim that areas MSTd and dlPFC form a functional sub-network together, on the basis of similarity in the fractions of neurons tuned to certain variables, and the distribution of the preferred value of some of these variables. However, the fractions of neurons tuned to the latent variables in MSTd and dlPFC (see Figure 2F) are actually quite different. Perhaps the authors could comment on this.Significance
The paper combines an original, naturalistic experimental design with an innovative method to analyse the data recorded during this design. As such, the paper opens up new avenues for primate research and I hope that this will inspire other primate researchers to go into similar directions.
-
Reviewer #3 (Public Review):
Noel et al provide a neural representational account of three brain areas in a virtual, visual navigation task paradigm especially designed to achieve a closed action-perception loop closely resembling natural behaviour. The authors recorded hundreds of neurons from three monkeys while the animals were engaged in the task where latent cognitive variables like distance travelled and distance to target continuously changed. The authors build on their previous work where they robustly characterized animal behaviour on this task paradigm. Here, they aim to find neural codes of dynamic, latent variables and report a mixed and heterogeneous profile of task variable coding distributed across the two brain areas in the parietal cortex (MSTd and area 7a) and one in the prefrontal cortex (dlPFC).
Major strength: …
Reviewer #3 (Public Review):
Noel et al provide a neural representational account of three brain areas in a virtual, visual navigation task paradigm especially designed to achieve a closed action-perception loop closely resembling natural behaviour. The authors recorded hundreds of neurons from three monkeys while the animals were engaged in the task where latent cognitive variables like distance travelled and distance to target continuously changed. The authors build on their previous work where they robustly characterized animal behaviour on this task paradigm. Here, they aim to find neural codes of dynamic, latent variables and report a mixed and heterogeneous profile of task variable coding distributed across the two brain areas in the parietal cortex (MSTd and area 7a) and one in the prefrontal cortex (dlPFC).
Major strength: Multi-area recording and the close-loop behavioural paradigm are major strengths of this study. The robust model-based analysis of neural data strengthens the paper even more. The correlation of coupling between MSTd and dlPFC and behaviour, albeit in a coarse time scale (of sessions), is particularly interesting and makes the paper strong by quantitatively relating behaviour to neural activity.
Major weakness: The paper mainly gives a long list of what task variables the three brain areas code for along with measures of connectivity between areas. Although this is a valuable contribution to the field, the study is not designed to test predictions of specific computational hypotheses. Towards the end of the paper, the authors bring up the two alternate mechanisms: vector-coding vs distance-coding, but only as a speculation. These two hypotheses could have been developed further at the outset to make specific predictions for neural dynamics and subsequently be tested in their data. This will likely lead to richer findings going beyond representations of task variables. Nevertheless, the findings presented in the paper are surely novel and exciting.
Impact: The main impact of the paper is neurophysiology under a novel, naturalistic behavioral paradigm. The data, both behavioral and neurophysiological, is rich and has potential to test predictions of more fine-grained computational hypotheses. However, the observation that MSTd codes for latent variables is not as surprising as the authors claim. Given the recent observations of heterogeneous variables represented in brain areas traditionally thought to be highly specific (e.g. locomotion variables in V1, mixed coding in EC etc.), it is not surprising to find latent variables in a 'traditionally' sensory area, especially in a continual behavioral paradigm where many variables are changing and are correlated.
Based on their previous work and this work, the authors mention multiple times the task strategy and its embodied nature. While the authors conclusively show the involvement of eye movement in solving the task, it is difficult to imagine a concrete definition of an embodied task strategy without clear alternate hypotheses. How would the animals behave if their eye movements were prevented? Worse performance (like humans did in their previous paper) or unable to perform (akin to a bird unable to fly without wings) or a different strategy? What should we predict based on the neural observation reported here? The impact of this paper would be greater if the authors bring up these questions and provide some speculations rooted in neurophysiological observations.
-
-