Contribution of linear and nonlinear mechanisms to predictive motion estimation
Curation statements for this article:-
Curated by eLife
eLife assessment
This paper will be of interest to sensory and computational neuroscientists. In it, the authors find maximally informative dimensions for primate retinal ganglion cells and use models based on these analyses to examine features of early visual processing that impact predictive coding of visual motion.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
Successful behavior relies on the ability to use information obtained from past experience to predict what is likely to occur in the future. A salient example of predictive encoding comes from the vertebrate retina, where neural circuits encode information that can be used to estimate the trajectory of a moving object. Predictive computations should be a general property of sensory systems, but the features needed to identify these computations across neural systems are not well understood. Here, we identify several properties of predictive computations in the primate retina that likely generalize across sensory systems. These features include calculating the derivative of incoming signals, sparse signal integration, and delayed response suppression. These findings provide a deeper understanding of how the brain carries out predictive computations and identify features that can be used to recognize these computations throughout the brain.
Article activity feed
-
eLife assessment
This paper will be of interest to sensory and computational neuroscientists. In it, the authors find maximally informative dimensions for primate retinal ganglion cells and use models based on these analyses to examine features of early visual processing that impact predictive coding of visual motion.
-
Reviewer #1 (Public Review):
In this paper, Liu et al. analyze a dataset of primate retinal ganglion cell responses to visual stimuli in order to find maximally informative dimensions in the inputs. They use models based on these analyses to examine features of early visual processing that influence predictive coding of visual motion in the early retina. This is an important set of questions because it remains unclear what principles drive sensory encoding and how those principles relate to circuit mechanisms found in sensory systems.
The strength in this paper lies in its rigorous analysis of the maximally informative dimensions (MIDs) of primate retinal ganglion cell signals, and the connections it makes between those dimensions and circuit models for retinal function.
The weakness of this paper lies in drawing strong connections …
Reviewer #1 (Public Review):
In this paper, Liu et al. analyze a dataset of primate retinal ganglion cell responses to visual stimuli in order to find maximally informative dimensions in the inputs. They use models based on these analyses to examine features of early visual processing that influence predictive coding of visual motion in the early retina. This is an important set of questions because it remains unclear what principles drive sensory encoding and how those principles relate to circuit mechanisms found in sensory systems.
The strength in this paper lies in its rigorous analysis of the maximally informative dimensions (MIDs) of primate retinal ganglion cell signals, and the connections it makes between those dimensions and circuit models for retinal function.
The weakness of this paper lies in drawing strong connections between those analyses and predictive coding by these cells. These analyses of predictive coding are interesting but not tightly related to the MID analysis. This paper also does little to address how the structure of the stimuli affect the conclusions they draw about what circuit features contribute to predictive coding of motion.
-
Reviewer #2 (Public Review):
Overall, I thoroughly enjoyed reading and reviewing this manuscript. I think that it contributes importantly to the literature and illustrates an appealing way to connect neural data to normative ideas, phenomenological models, and mechanic explanations. In particular, the suggestion that the retina is specifically tailored to support predictive information encoding is normatively appealing, because animals obtain ecological advantages by anticipating their environment. It would be very exciting to figure out how the retina accomplishes this task. The authors begin their analysis of this question by using spatiotemporal receptive fields to phenomenologically describe how retinal ganglion cells nonlinearly integrate visual signals presented in different regions of the visual field. This allows them to …
Reviewer #2 (Public Review):
Overall, I thoroughly enjoyed reading and reviewing this manuscript. I think that it contributes importantly to the literature and illustrates an appealing way to connect neural data to normative ideas, phenomenological models, and mechanic explanations. In particular, the suggestion that the retina is specifically tailored to support predictive information encoding is normatively appealing, because animals obtain ecological advantages by anticipating their environment. It would be very exciting to figure out how the retina accomplishes this task. The authors begin their analysis of this question by using spatiotemporal receptive fields to phenomenologically describe how retinal ganglion cells nonlinearly integrate visual signals presented in different regions of the visual field. This allows them to identify several spatiotemporal components of the receptive field, termed kernels, that contribute differentially to predictive information encoding. The authors then use neural circuit modeling to reproduce these receptive field properties using biologically plausible bipolar cell inputs to the retinal ganglion cells. This allows them to hypothesize how specific circuit properties may contribute to predictive information encoding. For example, the authors' current models allow them to address the roles of bipolar cell nonlinearities, spatially local coupling between bipolar cells, patchy bipolar cell to retinal ganglion cell connectivity, and activity-dependent neuronal adaptation.
By connecting predictive information encoding to receptive field properties and candidate circuit mechanisms, the authors hope to identify biological fingerprints of predictive information encoding that could carry over to other neural circuits in the brain. I did not find this component of the argument to be convincing. My main concern is that stimulus statistics and neuronal activity statistics dually contribute to the meaning of predictive information, but this study did not dissect the role of stimulus statistics at all. As a result, I think the paper places too much emphasis on mechanism, and not enough emphasis on natural sensory statistics. The authors do devote a figure to illustrating that their receptive field estimation procedure is insensitive to the stimulus ensemble used for fitting (Fig. 4). Indeed, perhaps the receptive field kernels would stay similar if they were fit to natural stimuli. However, it would still be the case that the pattern of predictive information encoding captured by these kernels would strongly vary as a function of stimulus ensemble. For example, here the authors use random synthetic stimuli with relatively short correlation times, which means that the temporal horizon for predictive information encoding is limited (see Liu et al., Nat Neuro, 2021). The pattern of predictive information encoding for natural stimuli may be very different, and it may be that different receptive field components and neural circuit mechanisms contribute to predictive information encoding in that context. Similarly, other sensory systems are adapted to process stimuli with other sensory statistics, and I do not think it's clear that the receptive field components and neural circuit mechanisms identified here will be universally relevant.
The manuscript uses information theoretic methods to infer multiple kernels that describe linear stimulus features that modulate spiking activity of retinal ganglion cells. A potentially interesting limitation of the study is that it assumes that "outputs of these kernels are summed prior to passing through a common nonlinearity." However, many other papers have found that neuronal activity is sometimes governed by multiple linear features that cannot be summed prior to their nonlinear action. It would be interesting to know whether these kinds of features contribute to predictive information encoding in the retina.
A major problem with the manuscript is that its methods are inadequately described. I think that a major revision will be required before readers will be able reproduce the manuscript's results. These missing methodological details also make it difficult for readers to fully assess the manuscript's conclusions, strengths, and limitations.
-
Reviewer #3 (Public Review):
This is a very interesting and sound work. It has been postulated that sensory neurons could optimize their information about future stimuli, but we still don't know how they can do that. This paper tackled this issue in depth with both phenomenological and mechanistic models, to understand which mechanisms could help optimize this predictive information, and show convincingly that several mechanisms can help for this.
The main limitation is that this is tested for motion at constant speed, and it would be interesting to know what happens in other cases. Also, the part about phenomenological modeling might need clarifications to understand better what really increases predictive information: it is clear the real system does it better than alternative, less realistic models, but in some cases it is not clear …
Reviewer #3 (Public Review):
This is a very interesting and sound work. It has been postulated that sensory neurons could optimize their information about future stimuli, but we still don't know how they can do that. This paper tackled this issue in depth with both phenomenological and mechanistic models, to understand which mechanisms could help optimize this predictive information, and show convincingly that several mechanisms can help for this.
The main limitation is that this is tested for motion at constant speed, and it would be interesting to know what happens in other cases. Also, the part about phenomenological modeling might need clarifications to understand better what really increases predictive information: it is clear the real system does it better than alternative, less realistic models, but in some cases it is not clear what is the key feature of the model.
-
