Contrary neuronal recalibration in different multisensory cortical areas
Curation statements for this article:-
Curated by eLife
eLife assessment
This important study combines rigorous behavior and single-unit recordings in nonhuman primates to investigate the role of three cortical areas in cross-modal sensory calibration, a form of neural plasticity that is important for perception and learning. The results convincingly demonstrate key similarities and striking differences across the three areas, and provide the first evidence for this form of calibration (in correspondence with behavior) at the level of single neurons. The work will be of broad interest to neuroscientists and psychologists studying multisensory perception, plasticity, and the role of sensory and association cortices in perceptual decisions.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
The adult brain demonstrates remarkable multisensory plasticity by dynamically recalibrating itself based on information from multiple sensory sources. After a systematic visual–vestibular heading offset is experienced, the unisensory perceptual estimates for subsequently presented stimuli are shifted toward each other (in opposite directions) to reduce the conflict. The neural substrate of this recalibration is unknown. Here, we recorded single-neuron activity from the dorsal medial superior temporal (MSTd), parietoinsular vestibular cortex (PIVC), and ventral intraparietal (VIP) areas in three male rhesus macaques during this visual–vestibular recalibration. Both visual and vestibular neuronal tuning curves in MSTd shifted – each according to their respective cues’ perceptual shifts. Tuning of vestibular neurons in PIVC also shifted in the same direction as vestibular perceptual shifts (cells were not robustly tuned to the visual stimuli). By contrast, VIP neurons demonstrated a unique phenomenon: both vestibular and visual tuning shifted in accordance with vestibular perceptual shifts. Such that, visual tuning shifted, surprisingly, contrary to visual perceptual shifts. Therefore, while unsupervised recalibration (to reduce cue conflict) occurs in early multisensory cortices, higher-level VIP reflects only a global shift, in vestibular space.
Article activity feed
-
-
Author Response
Reviewer #1 (Public Review):
Zeng and colleagues investigated the neural underpinnings of visual-vestibular recalibration. Specifically, they measured changes in three monkeys' perception of unisensory heading cues as well as associated changes in neuronal responses to these cues in three different cortical areas following prolonged exposure to systematic visual-vestibular discrepancies. Behavioral responses in a motion direction discrimination task indicate unisensory perceptual shifts in opposite directions that account for the cross-modal discrepancy the monkeys were exposed to. Neuronal firing patterns, related to motion discrimination judgments by means of neurometric functions indicated analogous shifts in neuronal tuning in areas MSTd and PIVC. In contrast, in area VIP tuning for visual heading stimuli shifted …
Author Response
Reviewer #1 (Public Review):
Zeng and colleagues investigated the neural underpinnings of visual-vestibular recalibration. Specifically, they measured changes in three monkeys' perception of unisensory heading cues as well as associated changes in neuronal responses to these cues in three different cortical areas following prolonged exposure to systematic visual-vestibular discrepancies. Behavioral responses in a motion direction discrimination task indicate unisensory perceptual shifts in opposite directions that account for the cross-modal discrepancy the monkeys were exposed to. Neuronal firing patterns, related to motion discrimination judgments by means of neurometric functions indicated analogous shifts in neuronal tuning in areas MSTd and PIVC. In contrast, in area VIP tuning for visual heading stimuli shifted in the same direction as tuning for vestibular stimuli and thus in contradiction to the observed perceptual shifts.
The shifts observed in MSTd and PIVC fit nicely with existing theories and results regarding cross-modal recalibration and substitute claims that activity in these areas might underlie perceptual decisions. The shift of visual tuning in VIP is surprising and will certainly spark further investigation.
Overall the results are really interesting, yet, the manuscript in its current form needs revisions along two dimensions, 1) data analysis and 2) writing.
We thank the reviewer for the positive comments and thoughtful suggestions, which have greatly helped us improve the data analysis and writing. Also, thank you for the thorough list of specific suggestions for improved writing and phrasing. This considerably helped us clarify these aspects in our manuscript.
Reviewer #2 (Public Review):
The manuscript by Zeng and colleagues aims to investigate how neural representations of sensory cues in two modalities (visual and vestibular) change when conflicts are introduced between the cues. The manuscript convincingly demonstrates that this recalibration process differs between areas MSTd (a multisensory region), where sensory responses recalibrated differently for visual and vestibular cues, following each modality's conflict, and area VIP ( a higher-level region), where responses follow the vestibular cue. More limited insights are present for area PIVC, where visual responses are limited.
The analyses generally support the conclusions of the authors, but I have two major suggestions to strengthen the statistical robustness of the manuscript:
- The analysis about the lack of visual recalibration in area PIVC would have been more convincing if the authors had used Bayesian statistics instead of regular t tests. In this way it would have been possible to estimate if the lack of visual recalibration in this area, for those few neurons that show visual tuning, can be taken as evidence for the absence of an effect or not. In the absence of this additional analysis, it is in fact difficult to properly interpret the results about area PIVC. Is PIVC more in line with MSTd, in view of the lack of visual responses? Or is there actually no visual recalibration, in contrast to both MSTd and VIP?
In response to this comment, we calculated the Bayesian Pearson correlation for visual recalibration in area PIVC, with the alternative hypothesis (H1) of a correlation between neuronal shifts and perceptual shifts and the null hypothesis (H0) of no correlation: Pearson's r = 0.26, and BF10 = 0.49. Thus, the evidence neither supports H1 nor H0. The lack of support for or against visual recalibration in PIVC primarily reflects the lack of robust tuning to visual heading stimuli in PIVC. Accordingly, in the manuscript, we do not argue for or against the recalibration of visual heading tuning in PIVC. Rather, we highlight that neurons in PIVC respond strongly to vestibular signals, but not so to visual heading stimuli and that the vestibular responses undergo recalibration. We agree that the lack of evidence for (or against) visual recalibration in PIVC primarily reflects the lack of robust tuning to visual heading stimuli. We interpret the observed shifts in vestibular tuning in PIVC as lower-level, sensory, recalibration (similar to MSTd) based on the broader understanding that PIVC encodes lower-level vestibular signals, with transient time-courses, and impoverished visual tuning (Chen et al., 2016; Chen et al., 2021). Our results are in line with this interpretation, and there is no reason to suspect that PIVC reflects more complex multisensory recalibration (like VIP). Nonetheless, the data could also be in line with alternative interpretations. Therefore, in the revised manuscript we now more explicitly explain this argument and have added limitations thereof, and alternative interpretations to the Discussion (in subsection “Limitations and future directions”, paragraph 2).
- For all statistical analyses, multi-level statistics would have been more appropriate than simple t-tests. In fact, since recordings come from few subjects, which in turn have relatively few recording sessions, there is a risk that the results are influenced by one subject and do not represent the full population. Admittedly, this is unlikely in view of the apparently large effect size and low p values. Nonetheless, a more appropriate statistical analysis would make the results more robust and convincing.
Thank you. We agree with this suggestion and have now: 1) added summary statistics for the individual monkeys, and 2) performed linear mixed model (LMM) analyses (please see our response to Essential Revisions Comment #1, for further details).
Once these issues are addressed, I believe that the manuscript would provide relevant evidence supporting the hypothesis that multisensory processing in the cortex is an area-specific phenomenon, and that effects observed in one area cannot be simply expected to operate elsewhere. This will therefore elucidate the mechanisms of multimodal plasticity.
Reviewer #3 (Public Review):
This study documents an empirical investigation of a fundamental brain process: adaptation to systematic cross-sensory discrepancies. The question is important, the experiment is carefully designed, and the results are striking. Following an unsupervised recalibration block, perceptual judgments of self-motion on the basis of visual and vestibular cues are systematically altered. These behavioral effects are mirrored by changes in the response properties of single neurons in areas MSTd and PIVC (provided that neurons in these areas exhibited selectivity for the sensory cue). Remarkably, neurons in downstream area VIP adjust their response properties in a very different manner, seemingly exclusively reflecting vestibular recalibration (which is opposite in direction to visual perceptual shifts). In the former two areas, the neural-behavior association follows the stimulus dynamics. In VIP, this association remains high beyond the life span of the stimulus. VIP typically exhibits strong choice signals. These decreased in strength after recalibration (an effect unique to area VIP). Together, these findings further dissociate VIP's functional role from that of MSTd and PIVC, without however, fully revealing what that role may be. These results offer a novel perspective on the neural basis of cross-sensory recalibration and will inspire future modeling studies of the neural basis of perception of self-motion.
We thank the reviewer for the supportive comments.
-
eLife assessment
This important study combines rigorous behavior and single-unit recordings in nonhuman primates to investigate the role of three cortical areas in cross-modal sensory calibration, a form of neural plasticity that is important for perception and learning. The results convincingly demonstrate key similarities and striking differences across the three areas, and provide the first evidence for this form of calibration (in correspondence with behavior) at the level of single neurons. The work will be of broad interest to neuroscientists and psychologists studying multisensory perception, plasticity, and the role of sensory and association cortices in perceptual decisions.
-
Reviewer #1 (Public Review):
Zeng and colleagues investigated the neural underpinnings of visual-vestibular recalibration. Specifically, they measured changes in three monkeys' perception of unisensory heading cues as well as associated changes in neuronal responses to these cues in three different cortical areas following prolonged exposure to systematic visual-vestibular discrepancies. Behavioral responses in a motion direction discrimination task indicate unisensory perceptual shifts in opposite directions that account for the cross-modal discrepancy the monkeys were exposed to. Neuronal firing patterns, related to motion discrimination judgments by means of neurometric functions indicated analogous shifts in neuronal tuning in areas MSTd and PIVC. In contrast, in area VIP tuning for visual heading stimuli shifted in the same …
Reviewer #1 (Public Review):
Zeng and colleagues investigated the neural underpinnings of visual-vestibular recalibration. Specifically, they measured changes in three monkeys' perception of unisensory heading cues as well as associated changes in neuronal responses to these cues in three different cortical areas following prolonged exposure to systematic visual-vestibular discrepancies. Behavioral responses in a motion direction discrimination task indicate unisensory perceptual shifts in opposite directions that account for the cross-modal discrepancy the monkeys were exposed to. Neuronal firing patterns, related to motion discrimination judgments by means of neurometric functions indicated analogous shifts in neuronal tuning in areas MSTd and PIVC. In contrast, in area VIP tuning for visual heading stimuli shifted in the same direction as tuning for vestibular stimuli and thus in contradiction to the observed perceptual shifts.
The shifts observed in MSTd and PIVC fit nicely with existing theories and results regarding cross-modal recalibration and substitute claims that activity in these areas might underlie perceptual decisions. The shift of visual tuning in VIP is surprising and will certainly spark further investigation.
Overall the results are really interesting, yet, the manuscript in its current form needs revisions along two dimensions, 1) data analysis and 2) writing.
-
Reviewer #2 (Public Review):
The manuscript by Zeng and colleagues aims to investigate how neural representations of sensory cues in two modalities (visual and vestibular) change when conflicts are introduced between the cues. The manuscript convincingly demonstrates that this recalibration process differs between areas MSTd (a multisensory region), where sensory responses recalibrated differently for visual and vestibular cues, following each modality's conflict, and area VIP ( a higher-level region), where responses follow the vestibular cue. More limited insights are present for area PIVC, where visual responses are limited.
The analyses generally support the conclusions of the authors, but I have two major suggestions to strengthen the statistical robustness of the manuscript:
The analysis about the lack of visual recalibration in …
Reviewer #2 (Public Review):
The manuscript by Zeng and colleagues aims to investigate how neural representations of sensory cues in two modalities (visual and vestibular) change when conflicts are introduced between the cues. The manuscript convincingly demonstrates that this recalibration process differs between areas MSTd (a multisensory region), where sensory responses recalibrated differently for visual and vestibular cues, following each modality's conflict, and area VIP ( a higher-level region), where responses follow the vestibular cue. More limited insights are present for area PIVC, where visual responses are limited.
The analyses generally support the conclusions of the authors, but I have two major suggestions to strengthen the statistical robustness of the manuscript:
The analysis about the lack of visual recalibration in area PIVC would have been more convincing if the authors had used Bayesian statistics instead of regular t tests. In this way it would have been possible to estimate if the lack of visual recalibration in this area, for those few neurons that show visual tuning, can be taken as evidence for the absence of an effect or not. In the absence of this additional analysis, it is in fact difficult to properly interpret the results about area PIVC. Is PIVC more in line with MSTd, in view of the lack of visual responses? Or is there actually no visual recalibration, in contrast to both MSTd and VIP?
For all statistical analyses, multi-level statistics would have been more appropriate than simple t-tests. In fact, since recordings come from few subjects, which in turn have relatively few recording sessions, there is a risk that the results are influenced by one subject and do not represent the full population. Admittedly, this is unlikely in view of the apparently large effect size and low p values. Nonetheless, a more appropriate statistical analysis would make the results more robust and convincing.
Once these issues are addressed, I believe that the manuscript would provide relevant evidence supporting the hypothesis that multisensory processing in the cortex is an area-specific phenomenon, and that effects observed in one area cannot be simply expected to operate elsewhere. This will therefore elucidate the mechanisms of multimodal plasticity.
-
Reviewer #3 (Public Review):
This study documents an empirical investigation of a fundamental brain process: adaptation to systematic cross-sensory discrepancies. The question is important, the experiment is carefully designed, and the results are striking. Following an unsupervised recalibration block, perceptual judgments of self-motion on the basis of visual and vestibular cues are systematically altered. These behavioral effects are mirrored by changes in the response properties of single neurons in areas MSTd and PIVC (provided that neurons in these areas exhibited selectivity for the sensory cue). Remarkably, neurons in downstream area VIP adjust their response properties in a very different manner, seemingly exclusively reflecting vestibular recalibration (which is opposite in direction to visual perceptual shifts). In the former …
Reviewer #3 (Public Review):
This study documents an empirical investigation of a fundamental brain process: adaptation to systematic cross-sensory discrepancies. The question is important, the experiment is carefully designed, and the results are striking. Following an unsupervised recalibration block, perceptual judgments of self-motion on the basis of visual and vestibular cues are systematically altered. These behavioral effects are mirrored by changes in the response properties of single neurons in areas MSTd and PIVC (provided that neurons in these areas exhibited selectivity for the sensory cue). Remarkably, neurons in downstream area VIP adjust their response properties in a very different manner, seemingly exclusively reflecting vestibular recalibration (which is opposite in direction to visual perceptual shifts). In the former two areas, the neural-behavior association follows the stimulus dynamics. In VIP, this association remains high beyond the life span of the stimulus. VIP typically exhibits strong choice signals. These decreased in strength after recalibration (an effect unique to area VIP). Together, these findings further dissociate VIP's functional role from that of MSTd and PIVC, without however, fully revealing what that role may be. These results offer a novel perspective on the neural basis of cross-sensory recalibration and will inspire future modeling studies of the neural basis of perception of self-motion.
-
