A quadratic model captures the human V1 response to variations in chromatic direction and contrast
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
This paper will be of interest to neuroscientists who study relationships between visual stimuli and their cortical representation, particularly, but not exclusively, those who use functional imaging techniques. The experiments are carefully designed, the dataset is substantial, and a model is presented that describes the data with very few parameters.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and Reviewer #2 agreed to share their names with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
An important goal for vision science is to develop quantitative models of the representation of visual signals at post-receptoral sites. To this end, we develop the quadratic color model (QCM) and examine its ability to account for the BOLD fMRI response in human V1 to spatially uniform, temporal chromatic modulations that systematically vary in chromatic direction and contrast. We find that the QCM explains the same, cross-validated variance as a conventional general linear model, with far fewer free parameters. The QCM generalizes to allow prediction of V1 responses to a large range of modulations. We replicate the results for each subject and find good agreement across both replications and subjects. We find that within the LM cone contrast plane, V1 is most sensitive to L-M contrast modulations and least sensitive to L+M contrast modulations. Within V1, we observe little to no change in chromatic sensitivity as a function of eccentricity.
Article activity feed
-
-
Author Response:
Reviewer #1 (Public Review):
This manuscript presents new data and a model that extend our understanding of color vision. The data are measurements of activity in human primary visual cortex in response to modulations of activity in the L- and M-cone photoreceptors. The model describes the data with impressive parsimony. This elegant simplification of a complex data set reveals a useful organizing principle of color processing in the visual cortex, and it is an important step towards construction of a model that predicts activity in the visual cortex to more complex visual patterns.
Strengths of the study include the innovative stimulus generation technique (which avoided technical artifacts that would have otherwise complicated data interpretation), the rigor of experimental design, the clear and even-handed data …
Author Response:
Reviewer #1 (Public Review):
This manuscript presents new data and a model that extend our understanding of color vision. The data are measurements of activity in human primary visual cortex in response to modulations of activity in the L- and M-cone photoreceptors. The model describes the data with impressive parsimony. This elegant simplification of a complex data set reveals a useful organizing principle of color processing in the visual cortex, and it is an important step towards construction of a model that predicts activity in the visual cortex to more complex visual patterns.
Strengths of the study include the innovative stimulus generation technique (which avoided technical artifacts that would have otherwise complicated data interpretation), the rigor of experimental design, the clear and even-handed data presentation, and the success of the QCM.
The study could be improved by a more thorough vetting of the QCM and additional discussion on the biological substrate of the activation patterns.
We thank the reviewer for the thoughtful summary of our work, for highlighting the strengths of our methodology and analysis, and for noting that our study will make a worthy contribution to understanding the organizing principles of visual cortex.
Reviewer #2 (Public Review):
The goal of this work is to advance knowledge of the neural bases of color perception. Color vision has been a model system for understanding how what we see arises from the coordinated action of neurons; detailed behavioral measurements revealed color vision's dependence upon three types of photoreceptors (trichromacy) and three second stage retinal circuits that compute sums and differences of the cone signals (color opponency). The processing of color at later, cortical stages has remained poorly understood however, and studies of human cortex have been hampered by methodologies that abandoned the detailed approach. Typical past work simply compared neural responses in two conditions, the presentation of colorful (formally, chromatic) vs grayscale (luminance) images. The present work returns to the older tradition that proved so successful.
The project's specific goals were to measure functional MRI responses in human cortex to a large range of colors, and equally importantly, capture the pattern responses with a quantitative model that can be used to predict response to many additional colors with just a few parameters. The reported work achieved these goals, establishing both a comprehensive data set and a modeling framework that together will provide a strong basis for future investigations. I would not hesitate to query the data further or to use the QCM model the paper provides to characterize other data sets.
The strengths of the work include its methodological rigor, which gives high confidence that the goals were achieved. Specifically:
- The visual presentation equipment was uniquely sophisticated, allowing it to correct for possible confounds due to differences in photoreceptor responses across the retina.
- The testing of the model was quite rigorous, aided by distinct replications of the experiment planned prior to data collection.
- The fMRI methods were also state of the art.
The work was well-situated within the literature, comparing its findings to past results. The limitations and assumptions of the present work were also clearly stated, and conclusions were not overstated.
Weaknesses of the current draft are relatively minor, however, I believe:
- The data could be presented in a way to make them more comparable to prior fMRI work, e.g. by using percent change units in more places, comparing the R^2 of model fits reported here to those reported in other papers, and explaining and exploring how the spatially uniform stimuli, used here but not in other fMRI studies, limited responses in visual areas beyond V1.
- Comparison between the two models, the GLM and QCM is not quite complete.
- The present results are not discussed in context with past results using EEG, and Brouwer and Heeger's model of fMRI responses to color.
- Implications of the basic pattern of response for the cortical neurons producing the data are discussed less than they could be.
We thank the reviewer for this clear summary of the paper, calling to attention our detailed approach to studying cortical color processing, and enthusiasm regarding the impact of our data and computational modeling.
Reviewer #3 (Public Review):
The authors describe a method for fitting a simple, separable function of contrast and cone excitation to a set of fMRI data generated from large, unstructured chromatic flicker stimuli that drive the L- and M- cone photoreceptors across a range of amplitudes and ratios. The function is of the form of a scaled ellipse – hereafter referred to as a 'Quadratic Color Model' (QCM). The QCM fits 6 parameters (ellipse orientation, ellipse elongation, and 4 parameters from a non-linear, saturating (Naka-Rushton) contrast response curve. The QCM fits the dataset well and the authors compare it (favorably) to a 40-parameter GLM that fits each separate combination of chromatic direction and contrast separately.
The authors note three things that 'did not have to be true' (and which are therefore interesting):
- The data are well-fit by a separable ellipse+contrast transducer - consistent with the idea that the underlying neuronal computations that process these stimuli combine relatively independent L-M and L+M contrast.
- The short axis of the QCM tends to align with the L-M cone contrast directing (indicating that this direction is one of maximum sensitivity and the L+M direction (long axis) is least sensitive. This finding is qualitatively consistent with psychophysical measurements of chromatic sensitivity.
- Fit parameters do not change much across the cortical surface – and in particular they are relatively constant with respect to eccentricity.
This is a technically solid paper – the data processing pipeline is meticulous, stimuli are tightly-calibrated (the ability to apply cone-isolating stimuli to fovea and periphery simultaneously is an impressive application of the 56-primary stimulus generator) and the authors have been careful to measure their stimuli before and after each experimental session. I have a few technical questions but I am completely satisfied that the authors are measuring what they think they are measuring.
The analysis, similarly, is exemplary in many ways. Robust fitting procedures are used and model performance and generalizablility are evaluated with a leave-run-out and leave-session-out cross validation procedures. Bootstrapped confidence intervals are generated for all fits and analysis code is available online.
The paper is also useful: it summarises a lot of (similar) previous findings in the fMRI color literature going back to the late 90s and points out that they can, in general, be represented with far fewer parameters than conditions. My main concerns are:
- Underlying mechanisms: The QCM is a convenient parameterization of low spatial-frequency, high temporal-frequency L-M responses. It will be a useful tool for future color vision researchers but I do not feel that I am learning very much that is new about human color vision. The choice to fit an ellipse to these data must have been motivated at least in part by inspection. It works in this case (possibly because of the particular combination of spatial and temporal frequencies that are probed) but it is not clear that this is a generic parametric model of human color responses in V1. Even very early fMRI data from stimuli with non-zero spatial frequency (for example, Engel, Zhang and Wandell '97) show response envelopes that are ellipse-like but which might well also have additional 'orthogonal' lobes or other oddities at some temporal frequencies.
- Model comparison: The 40-parameter GLM model provides a 'best possible' linear fit and gives a sense of the noisiness of the data but it feels a little like a strawman. It is possible to reduce the dimensionality of the fit significantly with the QCM but was it ever really plausible that the visual system would generate separate, independent responses for each combination of color direction and contrast? I suspect that given the fact that the response data are not saturating, it would be possible to replace the Naka-Rushton part of the model with a simple power function, reducing the parameter space even further. It would be more interesting to use the data to compare actual models of color processing in retina/V1 and, potentially, beyond V1.
- Link to perception. As the authors note, there is a rich history of psychophysics in this domain. The stimuli they choose are also, I think, well suited to modelling in the sense that they are likely to drive a very limited class of chromatic cells in V1 (those with almost no spatial frequency tuning). It is a shame therefore that no corresponding psychophysical data are presented to link physiology to perception. The issue is particularly acute because the stimulus differs from those typically used in more recent psychophysical experiments: it flickers relatively quickly and it has no spatial structure. It may, however, be more similar to the types of stimuli used prior to the advent of color CRTs : Maxwellian view systems that presented a single spot of light.
We thank the reviewer for their detailed comments on our paper and for highlighting our careful methodological approach and modeling of the data. We address the specific points.
-
Reviewer #3 (Public Review):
The authors describe a method for fitting a simple, separable function of contrast and cone excitation to a set of fMRI data generated from large, unstructured chromatic flicker stimuli that drive the L- and M- cone photoreceptors across a range of amplitudes and ratios. The function is of the form of a scaled ellipse – hereafter referred to as a 'Quadratic Color Model' (QCM). The QCM fits 6 parameters (ellipse orientation, ellipse elongation, and 4 parameters from a non-linear, saturating (Naka-Rushton) contrast response curve. The QCM fits the dataset well and the authors compare it (favorably) to a 40-parameter GLM that fits each separate combination of chromatic direction and contrast separately.
The authors note three things that 'did not have to be true' (and which are therefore interesting):
The data …
Reviewer #3 (Public Review):
The authors describe a method for fitting a simple, separable function of contrast and cone excitation to a set of fMRI data generated from large, unstructured chromatic flicker stimuli that drive the L- and M- cone photoreceptors across a range of amplitudes and ratios. The function is of the form of a scaled ellipse – hereafter referred to as a 'Quadratic Color Model' (QCM). The QCM fits 6 parameters (ellipse orientation, ellipse elongation, and 4 parameters from a non-linear, saturating (Naka-Rushton) contrast response curve. The QCM fits the dataset well and the authors compare it (favorably) to a 40-parameter GLM that fits each separate combination of chromatic direction and contrast separately.
The authors note three things that 'did not have to be true' (and which are therefore interesting):
The data are well-fit by a separable ellipse+contrast transducer - consistent with the idea that the underlying neuronal computations that process these stimuli combine relatively independent L-M and L+M contrast.
The short axis of the QCM tends to align with the L-M cone contrast directing (indicating that this direction is one of maximum sensitivity and the L+M direction (long axis) is least sensitive. This finding is qualitatively consistent with psychophysical measurements of chromatic sensitivity.
Fit parameters do not change much across the cortical surface – and in particular they are relatively constant with respect to eccentricity.
This is a technically solid paper – the data processing pipeline is meticulous, stimuli are tightly-calibrated (the ability to apply cone-isolating stimuli to fovea and periphery simultaneously is an impressive application of the 56-primary stimulus generator) and the authors have been careful to measure their stimuli before and after each experimental session. I have a few technical questions but I am completely satisfied that the authors are measuring what they think they are measuring.
The analysis, similarly, is exemplary in many ways. Robust fitting procedures are used and model performance and generalizablility are evaluated with a leave-run-out and leave-session-out cross validation procedures. Bootstrapped confidence intervals are generated for all fits and analysis code is available online.
The paper is also useful: it summarises a lot of (similar) previous findings in the fMRI color literature going back to the late 90s and points out that they can, in general, be represented with far fewer parameters than conditions. My main concerns are:
Underlying mechanisms: The QCM is a convenient parameterization of low spatial-frequency, high temporal-frequency L-M responses. It will be a useful tool for future color vision researchers but I do not feel that I am learning very much that is new about human color vision. The choice to fit an ellipse to these data must have been motivated at least in part by inspection. It works in this case (possibly because of the particular combination of spatial and temporal frequencies that are probed) but it is not clear that this is a generic parametric model of human color responses in V1. Even very early fMRI data from stimuli with non-zero spatial frequency (for example, Engel, Zhang and Wandell '97) show response envelopes that are ellipse-like but which might well also have additional 'orthogonal' lobes or other oddities at some temporal frequencies.
Model comparison: The 40-parameter GLM model provides a 'best possible' linear fit and gives a sense of the noisiness of the data but it feels a little like a strawman. It is possible to reduce the dimensionality of the fit significantly with the QCM but was it ever really plausible that the visual system would generate separate, independent responses for each combination of color direction and contrast? I suspect that given the fact that the response data are not saturating, it would be possible to replace the Naka-Rushton part of the model with a simple power function, reducing the parameter space even further. It would be more interesting to use the data to compare actual models of color processing in retina/V1 and, potentially, beyond V1.
Link to perception. As the authors note, there is a rich history of psychophysics in this domain. The stimuli they choose are also, I think, well suited to modelling in the sense that they are likely to drive a very limited class of chromatic cells in V1 (those with almost no spatial frequency tuning). It is a shame therefore that no corresponding psychophysical data are presented to link physiology to perception. The issue is particularly acute because the stimulus differs from those typically used in more recent psychophysical experiments: it flickers relatively quickly and it has no spatial structure. It may, however, be more similar to the types of stimuli used prior to the advent of color CRTs : Maxwellian view systems that presented a single spot of light.
-
Reviewer #2 (Public Review):
The goal of this work is to advance knowledge of the neural bases of color perception. Color vision has been a model system for understanding how what we see arises from the coordinated action of neurons; detailed behavioral measurements revealed color vision's dependence upon three types of photoreceptors (trichromacy) and three second stage retinal circuits that compute sums and differences of the cone signals (color opponency). The processing of color at later, cortical stages has remained poorly understood however, and studies of human cortex have been hampered by methodologies that abandoned the detailed approach. Typical past work simply compared neural responses in two conditions, the presentation of colorful (formally, chromatic) vs grayscale (luminance) images. The present work returns to the older …
Reviewer #2 (Public Review):
The goal of this work is to advance knowledge of the neural bases of color perception. Color vision has been a model system for understanding how what we see arises from the coordinated action of neurons; detailed behavioral measurements revealed color vision's dependence upon three types of photoreceptors (trichromacy) and three second stage retinal circuits that compute sums and differences of the cone signals (color opponency). The processing of color at later, cortical stages has remained poorly understood however, and studies of human cortex have been hampered by methodologies that abandoned the detailed approach. Typical past work simply compared neural responses in two conditions, the presentation of colorful (formally, chromatic) vs grayscale (luminance) images. The present work returns to the older tradition that proved so successful.
The project's specific goals were to measure functional MRI responses in human cortex to a large range of colors, and equally importantly, capture the pattern responses with a quantitative model that can be used to predict response to many additional colors with just a few parameters. The reported work achieved these goals, establishing both a comprehensive data set and a modeling framework that together will provide a strong basis for future investigations. I would not hesitate to query the data further or to use the QCM model the paper provides to characterize other data sets.
The strengths of the work include its methodological rigor, which gives high confidence that the goals were achieved. Specifically:
The visual presentation equipment was uniquely sophisticated, allowing it to correct for possible confounds due to differences in photoreceptor responses across the retina.
The testing of the model was quite rigorous, aided by distinct replications of the experiment planned prior to data collection.
The fMRI methods were also state of the art.
The work was well-situated within the literature, comparing its findings to past results. The limitations and assumptions of the present work were also clearly stated, and conclusions were not overstated.
Weaknesses of the current draft are relatively minor, however, I believe:
The data could be presented in a way to make them more comparable to prior fMRI work, e.g. by using percent change units in more places, comparing the R^2 of model fits reported here to those reported in other papers, and explaining and exploring how the spatially uniform stimuli, used here but not in other fMRI studies, limited responses in visual areas beyond V1.
Comparison between the two models, the GLM and QCM is not quite complete.
The present results are not discussed in context with past results using EEG, and Brouwer and Heeger's model of fMRI responses to color.
Implications of the basic pattern of response for the cortical neurons producing the data are discussed less than they could be.
-
Reviewer #1 (Public Review):
This manuscript presents new data and a model that extend our understanding of color vision. The data are measurements of activity in human primary visual cortex in response to modulations of activity in the L- and M-cone photoreceptors. The model describes the data with impressive parsimony. This elegant simplification of a complex data set reveals a useful organizing principle of color processing in the visual cortex, and it is an important step towards construction of a model that predicts activity in the visual cortex to more complex visual patterns.
Strengths of the study include the innovative stimulus generation technique (which avoided technical artifacts that would have otherwise complicated data interpretation), the rigor of experimental design, the clear and even-handed data presentation, and the …
Reviewer #1 (Public Review):
This manuscript presents new data and a model that extend our understanding of color vision. The data are measurements of activity in human primary visual cortex in response to modulations of activity in the L- and M-cone photoreceptors. The model describes the data with impressive parsimony. This elegant simplification of a complex data set reveals a useful organizing principle of color processing in the visual cortex, and it is an important step towards construction of a model that predicts activity in the visual cortex to more complex visual patterns.
Strengths of the study include the innovative stimulus generation technique (which avoided technical artifacts that would have otherwise complicated data interpretation), the rigor of experimental design, the clear and even-handed data presentation, and the success of the QCM.
The study could be improved by a more thorough vetting of the QCM and additional discussion on the biological substrate of the activation patterns.
-
Evaluation Summary:
This paper will be of interest to neuroscientists who study relationships between visual stimuli and their cortical representation, particularly, but not exclusively, those who use functional imaging techniques. The experiments are carefully designed, the dataset is substantial, and a model is presented that describes the data with very few parameters.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and Reviewer #2 agreed to share their names with the authors.)
-
