Multiple objects evoke fluctuating responses in several regions of the visual pathway
Curation statements for this article:-
Curated by eLife
eLife assessment
This important study adds to the growing body of evidence that neural responses fluctuate in time to alternatively represent one among multiple concurrent stimuli and that these fluctuations seize when objects fuse into one perceived object. The present study provides solid evidence from multiple brain areas and stimuli types to support this hypothesis. Overall, the study illustrates how the brain can use time dimension and synchrony to either parse or integrate stimuli into a coherent representation.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
How neural representations preserve information about multiple stimuli is mysterious. Because tuning of individual neurons is coarse (e.g., visual receptive field diameters can exceed perceptual resolution), the populations of neurons potentially responsive to each individual stimulus can overlap, raising the question of how information about each item might be segregated and preserved in the population. We recently reported evidence for a potential solution to this problem: when two stimuli were present, some neurons in the macaque visual cortical areas V1 and V4 exhibited fluctuating firing patterns, as if they responded to only one individual stimulus at a time (Jun et al., 2022). However, whether such an information encoding strategy is ubiquitous in the visual pathway and thus could constitute a general phenomenon remains unknown. Here, we provide new evidence that such fluctuating activity is also evoked by multiple stimuli in visual areas responsible for processing visual motion (middle temporal visual area, MT), and faces (middle fundus and anterolateral face patches in inferotemporal cortex – areas MF and AL), thus extending the scope of circumstances in which fluctuating activity is observed. Furthermore, consistent with our previous results in the early visual area V1, MT exhibits fluctuations between the representations of two stimuli when these form distinguishable objects but not when they fuse into one perceived object, suggesting that fluctuating activity patterns may underlie visual object formation. Taken together, these findings point toward an updated model of how the brain preserves sensory information about multiple stimuli for subsequent processing and behavioral action.
Article activity feed
-
-
-
Author response
Reviewer #1 (Public Review):
The main contribution appears to be related to functional specialization. I suggest clarifying the major novelty of the present report and to focus the introduction on it.
We thank this reviewer for this suggestion. We have revised the introduction to emphasize the functional specialization question. The changes are extensive; we have included a tracked-changes version of the manuscript to make these edits easy to see.
There is a growing literature on fluctuating neural firing patterns that is not considered in this report. The scholarship appears a bit impoverished with only 19 references, many of which point to work from this group of collaborators. I suggest that the authors consider the present work in the context of the wider literature more scholarly, even if not all the relations …
Author response
Reviewer #1 (Public Review):
The main contribution appears to be related to functional specialization. I suggest clarifying the major novelty of the present report and to focus the introduction on it.
We thank this reviewer for this suggestion. We have revised the introduction to emphasize the functional specialization question. The changes are extensive; we have included a tracked-changes version of the manuscript to make these edits easy to see.
There is a growing literature on fluctuating neural firing patterns that is not considered in this report. The scholarship appears a bit impoverished with only 19 references, many of which point to work from this group of collaborators. I suggest that the authors consider the present work in the context of the wider literature more scholarly, even if not all the relations of these different lines of work can be conclusively connected at this point. For a few examples, there is work by Kienitz and colleagues on fluctuating neural patterns in V4 evoked by competing grating stimuli. Also, the work by Engel, Moore, and colleagues on 'on' and 'off' states in the context of selective attention seems relevant, or the work by Fiebelkorn and Kastner on rhythmic perception and attention.
We agree completely with this suggestion! We have reworded the introduction to be more inclusive of other research in this area (especially Kienitz and colleagues – exciting work that we are pleased to have had brought to our attention) and we have added about 500 words in the Discussion to cover the work on on/off states (Engel et al.), rhythmic perception (Fiebelkorn & Kastner and others), and attention more generally (e.g., Triesman & Gelade’s work on serial sampling). We are particularly pleased to add these sections because these topics are very much on our minds – we have a commentary piece under review elsewhere in which we evaluate these synergistic lines of approach in a more complete fashion. In total, we’ve added about 15 additional references.
Reviewer #2 (Public Review):
The description of the results would benefit from a better explanation of how low spike counts may influence the outcome of the analysis. Due to a smoothing procedure used for visualization, the spike counts for the paired stimuli (AB, black lines) shown in Figure 3a-b and Figure 4a-d go below 0. However, the actual spike count on a trial can not go below 0. The symmetric smoothing procedure may hide an underlying skewed distribution of spike counts that can only be positive. The statistical analysis is not performed on the smoothed distribution but on the actual spike counts, and the validity of the result is therefore not in question. However, the paper would benefit from 1) visualization of the unsmoothed trial counts, and 2) an explanation of how assumptions of symmetric/skewed distributions may affect the outcome.
We thank the reviewers for noting this and making these suggestions. We now include unsmoothed raw spike counts in all the example figures (Figure 3a-b and Figure 4a-d). With regard to the symmetric/skewed distributions and the analysis methods, a Poisson distribution will be skewed at low rates and become more symmetric at higher rates, so this is already incorporated into the analysis. Indeed, the utility of Poisson distributions for fitting non-negative data is one of the reasons these distributions are so commonly used in neuroscience. We now make this point explicitly at the beginning of Methods/Data analysis: “Our method centers on modeling spike counts based on Poisson distributions, a common technique for handling non-negative count data in neuroscience and other fields.” With this edit as well as the revised example figures now making clear that no spike counts are below zero, we are optimistic that readers will better understand the analysis method and how the shape of response distributions are incorporated into it.
-
eLife assessment
This important study adds to the growing body of evidence that neural responses fluctuate in time to alternatively represent one among multiple concurrent stimuli and that these fluctuations seize when objects fuse into one perceived object. The present study provides solid evidence from multiple brain areas and stimuli types to support this hypothesis. Overall, the study illustrates how the brain can use time dimension and synchrony to either parse or integrate stimuli into a coherent representation.
-
Reviewer #1 (Public Review):
The study by Schmehl and colleagues asks an important question, i.e. how are multiple objects/stimuli represented in the visual system despite broad tuning properties of neurons along multiple different dimensions (e.g. space, features). This is a continuation of an impactful and highly significant line of work from the Groh lab and their collaborators. In previous work, they showed that fluctuations in firing patterns may be critical in representing multiple objects and parse them in time. In this particular study, the authors ask three specific questions to extend these observations: (i) Are such fluctuations widespread in the visual system?; (ii) Are they related to the perceptual distinction of objects?; (iii) And how are they related to the functional specialization of neuronal populations along feature …
Reviewer #1 (Public Review):
The study by Schmehl and colleagues asks an important question, i.e. how are multiple objects/stimuli represented in the visual system despite broad tuning properties of neurons along multiple different dimensions (e.g. space, features). This is a continuation of an impactful and highly significant line of work from the Groh lab and their collaborators. In previous work, they showed that fluctuations in firing patterns may be critical in representing multiple objects and parse them in time. In this particular study, the authors ask three specific questions to extend these observations: (i) Are such fluctuations widespread in the visual system?; (ii) Are they related to the perceptual distinction of objects?; (iii) And how are they related to the functional specialization of neuronal populations along feature dimensions (e.g. faces, motion).
It seems to me that there is ample evidence for the first two questions from previous work by these authors. For (i), fluctuations in firing patterns related to multiple stimuli have been shown in the auditory (e.g. inferior colliculus, Caruso et al., 2018) and multiple areas of the visual system (i.e. V1, V4, and the face patch system; Caruso et al., 2018; Jun et al., 2022). The present study adds data from MT to this increasing evidence. For (ii), Jun et al., 2022 already showed that fluctuations are not related to stimuli perceived as merged, or not distinct. Thus, the main contribution appears to be related to functional specialization. I suggest clarifying the major novelty of the present report and to focus the introduction on it.
The present work analyzed three different data sets acquired in different areas (V1, V4, MT, IT face network), using different feature stimuli (motion, faces), obtained under various attention conditions/states (passive fixation, actively ignored). Many of the results are nice confirmations and minor extensions of previous work. The conceptual advance and novelty of the findings are therefore limited.
There is a growing literature on fluctuating neural firing patterns that is not considered in this report. The scholarship appears a bit impoverished with only 19 references, many of which point to work from this group of collaborators. I suggest that the authors consider the present work in the context of the wider literature more scholarly, even if not all the relations of these different lines of work can be conclusively connected at this point. For a few examples, there is work by Kienitz and colleagues on fluctuating neural patterns in V4 evoked by competing grating stimuli. Also, the work by Engel, Moore, and colleagues on 'on' and 'off' states in the context of selective attention seems relevant, or the work by Fiebelkorn and Kastner on rhythmic perception and attention.
-
Reviewer #2 (Public Review):
In a beautiful line of work, the authors have proposed the intriguing idea that activity patterns of neurons can fluctuate between representing one of multiple stimuli in its receptive field. This allows for time-multiplexing of information by neural populations. The idea was initially proposed by Caruso et al (2018) and tested for both auditory and visual stimuli and later extended in Jun et al (2022). The current study analyzes additional datasets to further extend the conclusions across multiple areas and different stimulus sets.
Together with the earlier work, the current study provides solid evidence for the hypothesis that fluctuating activity patterns in neurons representing multiple stimuli may be a general phenomenon. This exciting possibility may have implications for the studies of perception, …
Reviewer #2 (Public Review):
In a beautiful line of work, the authors have proposed the intriguing idea that activity patterns of neurons can fluctuate between representing one of multiple stimuli in its receptive field. This allows for time-multiplexing of information by neural populations. The idea was initially proposed by Caruso et al (2018) and tested for both auditory and visual stimuli and later extended in Jun et al (2022). The current study analyzes additional datasets to further extend the conclusions across multiple areas and different stimulus sets.
Together with the earlier work, the current study provides solid evidence for the hypothesis that fluctuating activity patterns in neurons representing multiple stimuli may be a general phenomenon. This exciting possibility may have implications for the studies of perception, attention, decision-making, and other cognitive functions.
In the current study, the claim that the fluctuating activity patterns may be a general phenomenon is supported by multiple data sets from area MT and face patches MF and AL in IT cortex, using multiple stimulus sets (moving dots and gratings for MT, and face-face and face-object pairs for IT cortex). The major strength of this study is the consistency of the results across these areas and stimulus sets.
The description of the results would benefit from a better explanation of how low spike counts may influence the outcome of the analysis. Due to a smoothing procedure used for visualization, the spike counts for the paired stimuli (AB, black lines) shown in Figure 3a-b and Figure 4a-d go below 0. However, the actual spike count on a trial can not go below 0. The symmetric smoothing procedure may hide an underlying skewed distribution of spike counts that can only be positive. The statistical analysis is not performed on the smoothed distribution but on the actual spike counts, and the validity of the result is therefore not in question. However, the paper would benefit from 1) visualization of the unsmoothed trial counts, and 2) an explanation of how assumptions of symmetric/skewed distributions may affect the outcome.
Overall, the authors have presented an interesting hypothesis that is supported by rigorous analysis, they clearly described the results, and they have given a fair discussion of what we can and cannot conclude from this dataset. This line of work deserves the attention of a broad audience within the field of neuroscience.
-