Coding of chromatic spatial contrast by macaque V1 neurons
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
This paper is of broad interest to psychologists, neuroscientists, and engineers who seek to understand how color information is represented in visual cortex. The experiments provide sharply focussed tests of how chromatic information is compared across different spatial locations by individual neurons in visual cortex. The experiments are sound and the results speak to fundamental principles of encoding.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and reviewer #2 agreed to share their names with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
Color perception relies on comparisons between adjacent lights, but how the brain performs these comparisons is poorly understood. To elucidate the underlying mechanisms, we recorded spiking responses of individual V1 neurons in macaque monkeys to pairs of stimuli within the classical receptive field (RF). We estimated the spatial-chromatic RF of each neuron and then presented customized colored edges using a closed-loop technique. We found that many double-opponent (DO) cells, which have spatially and chromatically opponent RFs, responded to chromatic contrast as a weighted sum, akin to how other V1 neurons responded to luminance contrast. Yet other neurons integrated chromatic signals nonlinearly, confirming that linear signal integration is not an obligate property of V1 neurons. The functional similarity of cone-opponent DO cells and cone non-opponent simple cells suggests that these two groups may share a common underlying circuitry, promotes the construction of image-computable models for full-color image representation, and sheds new light on V1 complex cells.
Article activity feed
-
-
Author Response
Reviewer #1 (Public Review):
There are very few studies on the spatial integration of color signals of V1 receptive fields, which is a striking gap in knowledge given the importance of color to primate vision and the powerfulness that spatial analysis of luminance contrast integration has proven for understanding how V1 works. This paper helps fill this major gap in knowledge. The main take home is that double opponent cells and simple cells are more likely to be linear in how they integrate signals across their receptive fields than a sample of non-double-opponent/non-simple cells. This conclusion is consistent with the limited data presently in the literature, and I wonder if further analysis of the rich dataset could uncover some deeper insights.
We thank the reviewer for highlighting the gap in knowledge that our …
Author Response
Reviewer #1 (Public Review):
There are very few studies on the spatial integration of color signals of V1 receptive fields, which is a striking gap in knowledge given the importance of color to primate vision and the powerfulness that spatial analysis of luminance contrast integration has proven for understanding how V1 works. This paper helps fill this major gap in knowledge. The main take home is that double opponent cells and simple cells are more likely to be linear in how they integrate signals across their receptive fields than a sample of non-double-opponent/non-simple cells. This conclusion is consistent with the limited data presently in the literature, and I wonder if further analysis of the rich dataset could uncover some deeper insights.
We thank the reviewer for highlighting the gap in knowledge that our study helps to fill and for the excellent suggestions for ways to improve the manuscript. In response to both reviewers, we have conducted new analyses that uncover deeper insights into signal integration in V1. These new analyses have been incorporated into the revised manuscript.
Reviewer #2 (Public Review):
De and Horwitz deploy a focussed technique for testing the linearity of spatial summation for V1 neurons with spatial opponency, with the emphasis being on the properties of cells that encode chromatic information in a spatially opponent manner - so called double opponent cells. The technique isolates non-linearities of summation from non-linearities that occur after summation, by using an adaptive procedure to home in on stimulus contrasts in different color directions that produce a pre-defined criterion response. The authors conclude that many (but not all) double opponent cells embody linear spatial summation, and discuss implications for our understanding of the cortical circuitry that mediates color vision. The data appear carefully collected and generally well-analyzed. There are some points, elaborated in broad strokes below, where I think the paper would benefit from further elaboration of the data and its implications, and the paper would also benefit from some revisions to improve clarity.
How are results affected by the cell classification criteria? The authors apply criteria to sort cells into four classes: simple, double opponent, NSNDO, and those not studied further. Response properties are then studied as a function of cell class. Criteria for classification include presence/absence of spatial opponency revealed by the pixel white noise measurements and the adequacy of a linear STA to describe the hyperpixel white noise data. I think more work is needed to clarify for the reader the extent to which these criteria, in and of themselves, affect the results for each class studied. In particular, if a linear STA describes the hyperpixel white noise data, shouldn't we then expect to find linear summation in the spatial receptive field in that same hyperpixel white noise data? I understand, as the authors point out, that the Phase 3 measurements could reveal failures of spatial summation not seen in the hyperpixel white noise data. But I'm a bit perplexed by the outliers in the NLI indices in Figure 3D. What properties of these cells allow a linear 6D STA to handle the hyperpixel white noise data well, but cause them to summate over space non-linearly for that same hyperpixel white noise data? In terms of the new information provided by the Phase 3 measurements, I wasn't able to get a sense of how much harder these stimuli were driving the cells than the Phase 2 measurements. It seemed like this was the intent of Figure 2 - Figure Supplement 1 and Figure 3 - Figure Supplment 1, but those two figures in the end didn't provide this information in a manner I could digest. Absent this, it was hard to tell how much more we are learning from the Phase 3 data. Could the higher NLI's here than in Phase 2 be a consequence of some stimuli but not others driving the neuron into saturation? And although the authors write on page 15 "Nevertheless, we found that nonlinearities detected in Phase 2 of our experiment were a good indicator of nonlinearity over the greater stimulus duration and range of contrasts in Phase 3, principally for the NSNDO cells (Figure 3E)", those correlations look very weak to me. I was left hoping for a better understanding the commonalities and differences in the data between Phases 2 and 3. I'm also not sure of the reliability of the measured NLI's for each cell with each method. Can anything more be provided about that? I note here that I did study the section of the discussion that nominally addresses some if these issues, and that my comments above remain after that study.
The Reviewer brings up several important points that are addressed individually below.
The revised manuscript is more explicit about the role of the cell classification criteria on the results. Particular emphasis is placed on the role of the spike-triggered covariance criterion in enriching the pools of simple cells and DO cells with neurons that are approximately linear.
We agree with the Reviewer that, if a linear STA describes the hyperpixel white noise data well, we expect to find linear summation in the spatial receptive field in that same hyperpixel white noise data analyzed in other ways. A critical question is “does the STA describe the white noise data well?”. We address this question in two ways in this report: with an analysis of (the statistical significance of) the first principal component of the hyperpixel spike-triggering stimuli (PC1) and with a comparison of GLM and GQM fits to the hyperpixel white noise data (the white noise NLI). These analyses are related but are sensitive to different types of departure from linearity.
Consider a neuron whose output is the product of two half-wave rectified linear subunits (see Figure 2 – Figure Supplement 5). Such a neuron would have a large white noise NLI due to the non-linear interaction between the subfields, but it would lack a significant PC1, because the nonlinearity tightens the distribution of excitatory stimuli, and the PC1 is the dimension along which the stimulus distribution is widest. In principle, such a nonlinearity would manifest in the smallest principal component, but in practice, small PCs often resemble the STA, which complicates their interpretation.
Conversely, a neuron can have a significant PC1 but a small NLI. For example, consider a neuron that has a half-wave rectified response to modulations of one color channel but a full-wave rectified response to another. Such a neuron will have a significant PC1 due to the full-wave rectification, but an NLI near zero, because this nonlinearity is hidden once the stimuli are projected onto the STA (recall that the white noise NLI is computed from a pair of 1-D projections not the original 6-D representation). Code simulating these hypothetical neurons (used to produce Figure 2 – Figure Supplement 5) is available at GitHub (https://github.com/horwitzlab/Chromatic_spatial_contrast).
The original submission lacked documentation of the difference in firing rates produced during Phases 2 and 3. We have added a new supplementary figure that quantifies this difference (see Figure 2 – Figure Supplement 2). Figure 2 – Figure Supplement 1 illustrates the range of inputs provided in Phases 2 & 3. This has been clarified in the revised text.
Please note that the data shown in Figure 3D are isoresponse NLIs (that is, NLIs computed from responses recorded during Phase 3 of the experiment) not white noise NLIs (NLIs computed from the hyperpixel white noise shown during Phase 2 of the experiment). This has been clarified in the revised text.
We agree that the correlation between the white noise NLI and isoresponse NLI measurements is weak. A full treatment of the differences in neural responses to the stimuli presented in Phase 2 & 3 is beyond the scope of this study. Nevertheless, we can think of several reasons that some neurons may have appeared more nonlinear in Phase 3 than they did in Phase 2. The first is, as suggested above, Phase 3 stimuli had higher contrast than Phase 2 stimuli, and are more likely to have engaged nonlinear gain control mechanisms upstream or within V1. Second, the linear and nonlinear models in Phase 2 had 3 and 6 parameters, respectively, but 2 and 5 in Phase 3, and this may affect the ratio of prediction errors. Third, nonstationary responses are expected to affect isoresponse NLIs more severely than white noise NLIs, because of the sequential way that isoresponse points were measured in Phase 3.
Assessing the reliability of NLIs within cells is challenging because of the crossvalidation that is built into the definition. To address this comment, we used a jackknife procedure that quantifies the spread of NLIs computed from each of the data partitions used in the cross-validation.
Implications of the results for models. As the authors summarize in their introduction, the motivation for testing the linearity of spatial summation is that the results can guide how we formulate response models for V1 chromatically sensitive cells. More discussion of this would be helpful. As an example, could cells with the nonlinear spatial filtering as shown in Figure 1C be classified as DO, making them relevant to the focussed tests applied in this paper? Or are they necessarily NSNDO? More generally, can the authors spend a little time discussing what classes of response models they would pursue for DO cells that do/don't show linear spatial summation, and for NSNDO cells that do/don't show linear spatial summation. Such discussion would tie the results of the primary data back to the motivating question in a more satisfactory manner, I think. Such discussion could also be used as a vehicle to discuss what the authors think about the DO cells that fail to show linear spatial summation and the NSNDO cells that do, something I found under-treated in the results. As with the comment above, I did read the sections of the paper that speak to this question, but still find it that it would benefit from going deeper.
Inspired by this comment, we have added a new section to the Results that considers response models for neurons that do not show linear spatial summation. Specifically, we test the model illustrated in Figure 1C and reject it for many neurons. Figure 1C depicts a neuron that integrates inputs linearly within each subfield but nonlinearly across subfields. Within each RF subfield, therefore, this neuron conforms to a linear-nonlinear cascade model. Critically, during Phase 2 of the experiment, the stimulation at one RF subfield can be considered as an additive noise with respect to the signal generated by the other RF subfield. This is because the influences of the two RF subfields combine additively (under the model) and the modulations of the two hyperpixels are independent.
To test this model, we compared GLM and GQM fits as we did in the analysis of the white noise NLI. The regressors in this analysis were the modulations of the three color channels from a single subfield. These GLMs fit the data systematically worse than GQMs as assessed by cross-validated prediction error. This result indicates that the nonlinearity of the NSNDO cells is unlikely to be a result of nonlinear combination of inputs from two linear RF subfields, as postulated by the model in Figure 1C. Instead, for many NSNDO neurons the nonlinearity appears to arise from nonlinear combinations of signals within individual subfields. We mention in the Discussion that linear DO cells may lie on a continuum with some NSNDO cells.
Color properties of subfields. The study measures detailed properties of cells that show at least two distinct subfields in the initial pixel white noise analysis. The paper focuses on whether signals from such subfields are combined linearly before any downstream linearities. However, there is another feature of the data that seems central to understanding these cells, and that is what the chromatic properties of these subfields are, and how strong in the data the constraint that the chromatic properties of the two separate subfields be complementary is. It is stated in passing (page 7) that "the two sides of the hyper pixel STA were complementary or nearly so", but it would be nice to see this treated in more detail and also to understand whether there are differences in the distribution of the chromatic properties of the two sides between the DO and NSNDO cells, and between cells with low and high non-linearity indices.
We have added new section on the chromatic properties of the subfields of the neurons we studied (Figure 2 – Figure Supplement 3).
-
Reviewer #2 (Public Review):
De and Horwitz deploy a focussed technique for testing the linearity of spatial summation for V1 neurons with spatial opponency, with the emphasis being on the properties of cells that encode chromatic information in a spatially opponent manner - so called double opponent cells. The technique isolates non-linearities of summation from non-linearities that occur after summation, by using an adaptive procedure to home in on stimulus contrasts in different color directions that produce a pre-defined criterion response. The authors conclude that many (but not all) double opponent cells embody linear spatial summation, and discuss implications for our understanding of the cortical circuitry that mediates color vision. The data appear carefully collected and generally well-analyzed. There are some points, …
Reviewer #2 (Public Review):
De and Horwitz deploy a focussed technique for testing the linearity of spatial summation for V1 neurons with spatial opponency, with the emphasis being on the properties of cells that encode chromatic information in a spatially opponent manner - so called double opponent cells. The technique isolates non-linearities of summation from non-linearities that occur after summation, by using an adaptive procedure to home in on stimulus contrasts in different color directions that produce a pre-defined criterion response. The authors conclude that many (but not all) double opponent cells embody linear spatial summation, and discuss implications for our understanding of the cortical circuitry that mediates color vision. The data appear carefully collected and generally well-analyzed. There are some points, elaborated in broad strokes below, where I think the paper would benefit from further elaboration of the data and its implications, and the paper would also benefit from some revisions to improve clarity.
How are results affected by the cell classification criteria? The authors apply criteria to sort cells into four classes: simple, double opponent, NSNDO, and those not studied further. Response properties are then studied as a function of cell class. Criteria for classification include presence/absence of spatial opponency revealed by the pixel white noise measurements and the adequacy of a linear STA to describe the hyperpixel white noise data. I think more work is needed to clarify for the reader the extent to which these criteria, in and of themselves, affect the results for each class studied. In particular, if a linear STA describes the hyperpixel white noise data, shouldn't we then expect to find linear summation in the spatial receptive field in that sane hyperspectral white noise data? I understand, as the authors point out, that the Phase 3 measurements could reveal failures of spatial summation not seen in the hyperpixel white noise data. But I'm a bit perplexed by the outliers in the NLI indices in Figure 3D. What properties of these cells allow a linear 6D STA to handle the hyperpixel white noise data well, but cause them to summate over space non-linearly for that same hyperpixel white noise data? In terms of the new information provided by the Phase 3 measurements, I wasn't able to get a sense of how much harder these stimuli were driving the cells than the Phase 2 measurements. It seemed like this was the intent of Figure 2 - Figure Supplement 1 and Figure 3 - Figure Supplment 1, but those two figures in the end didn't provide this information in a manner I could digest. Absent this, it was hard to tell how much more we are learning from the Phase 3 data. Could the higher NLI's here than in Phase 2 be a consequence of some stimuli but not others driving the neuron into saturation? And although the authors write on page 15 "Nevertheless, we found that nonlinearities detected in Phase 2 of our experiment were a good indicator of nonlinearity over the greater stimulus duration and range of contrasts in Phase 3, principally for the NSNDO cells (Figure 3E)", those correlations look very weak to me. I was left hoping for a better understanding the commonalities and differences in the data between Phases 2 and 3. I'm also not sure of the reliability of the measured NLI's for each cell with each method. Can anything more be provided about that? I note here that I did study the section of the discussion that nominally addresses some if these issues, and that my comments above remain after that study.
Implications of the results for models. As the authors summarize in their introduction, the motivation for testing the linearity of spatial summation is that the results can guide how we formulate response models for V1 chromatically sensitive cells. More discussion of this would be helpful. As an example, could cells with the non-linear spatial filtering as shown in Figure 1C be classified as DO, making them relevant to the focussed tests applied in this paper? Or are they necessarily NSNDO? More generally, can the authors spend a little time discussing what classes of response models they would pursue for DO cells that do/don't show linear spatial summation, and for NSNDO cells that do/don't show linear spatial summation. Such discussion would tie the results of the primary data back to the motivating question in a more satisfactory manner, I think. Such discussion could also be used as a vehicle to discuss what the authors think about the DO cells that fail to show linear spatial summation and the NSNDO cells that do, something I found under-treated in the results. As with the comment above, I did read the sections of the paper that speak to this question, but still find it that it would benefit from going deeper.
Color properties of subfields. The study measures detailed properties of cells that show at least two distinct subfields in the initial pixel white noise analysis. The paper focuses on whether signals from such subfields are combined linearly before any downstream linearities. However, there is another feature of the data that seems central to understanding these cells, and that is what the chromatic properties of these subfields are, and how strong in the data the constraint that the chromatic properties of the two separate subfields be complementary is. It is stated in passing (page 7) that "the two sides of the hyper pixel STA were complementary or nearly so", but it would be nice to see this treated in more detail and also to understand whether there are differences in the distribution of the chromatic properties of the two sides between the DO and NSNDO cells, and between cells with low and high non-linearity indices.
-
Reviewer #1 (Public Review):
There are very few studies on the spatial integration of color signals of V1 receptive fields, which is a striking gap in knowledge given the importance of color to primate vision and the powerfulness that spatial analysis of luminance contrast integration has proven for understanding how V1 works. This paper helps fill this major gap in knowledge. The main take home is that double opponent cells and simple cells are more likely to be linear in how they integrate signals across their receptive fields than a sample of non-double-opponent/non-simple cells. This conclusion is consistent with the limited data presently in the literature, and I wonder if further analysis of the rich dataset could uncover some deeper insights.
-
Evaluation Summary:
This paper is of broad interest to psychologists, neuroscientists, and engineers who seek to understand how color information is represented in visual cortex. The experiments provide sharply focussed tests of how chromatic information is compared across different spatial locations by individual neurons in visual cortex. The experiments are sound and the results speak to fundamental principles of encoding.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and reviewer #2 agreed to share their names with the authors.)
-
