Expansion and contraction of resource allocation in sensory bottlenecks

Article activity feed

1. 

   eLife

   Aug 10, 2022

   Author Response

   Reviewer #1 (Public Review):

   Edmondson et al. develop an efficient coding approach to study resource allocation in resource constrained sensory systems, with a particular focus on somatosensory representations. Their approach is based on a simple, yet novel insight. Namely - to achieve output decorrelation when encoding stimuli from regions with different input statistics, neurons in the sensory bottleneck should be allocated to these regions according to jointly sorted eigenvalues of the input covariance matrix. The authors demonstrate that, even in a simple scenario, this allocation scheme leads to a complex, non-monotonic relationship between the number of neurons representing each region, receptor density and input statistics. To demonstrate the utility of their approach, the authors generate predictions about …

   Author Response

   Reviewer #1 (Public Review):

   Edmondson et al. develop an efficient coding approach to study resource allocation in resource constrained sensory systems, with a particular focus on somatosensory representations. Their approach is based on a simple, yet novel insight. Namely - to achieve output decorrelation when encoding stimuli from regions with different input statistics, neurons in the sensory bottleneck should be allocated to these regions according to jointly sorted eigenvalues of the input covariance matrix. The authors demonstrate that, even in a simple scenario, this allocation scheme leads to a complex, non-monotonic relationship between the number of neurons representing each region, receptor density and input statistics. To demonstrate the utility of their approach, the authors generate predictions about cortical representations in the star-nosed mole, and observe a close match between theory and data.

   Strengths:

   These results are certainly interesting and address an issue which to my knowledge has not been studied in-depth before. Touch is a sensory modality rarely mentioned in theoretical studies of sensory coding, and this work contributes to this direction of research.

   A clear strength of the paper is that it demonstrates the existence of non-trivial dependence between resource allocation, bottleneck size and input statistics. Discussion of this relationship highlights the importance of nuance and subtlety in theoretical predictions in neuroscience.

   The proposed theory can be applied to interpret experimental observations - as demonstrated with the example of the star-nosed mole. The prediction of cortical resource allocation is a close match to experimental data.

   We thank the reviewer for the feedback. Indeed, demonstrating an ‘interesting’ effect in even such a simple model was one of the main aims.

   Weaknesses:

   The central weakness of this work are the strong assumptions which are not clearly stated. In result, the consequences of these assumptions are not discussed in sufficient depth which may limit the generality of the proposed approach. In particular:

   1. The paper focuses on a setting with vanishing input noise, where the efficient coding strategy is toreduce the redundancy of the output (for example through decorrelation). This is fine, however, it is not a general efficient coding solution as indicated in the introduction - it is a specific scenario with concrete assumptions, which should be clearly discussed from the beginning.

   1. The model assumes that the goal of the system is to generate outputs, whose covariance structure isan identity matrix (Eq. 1). This corresponds to three assumptions: a) variances of output neurons are equalized, b) the total amount of output variance is equal to M (i.e. the number of of output neurons), c) the activity of output neurons is decorrelated. The paper focuses only on the assumption c), and does not discuss consequences or biological plausibility of assumptions a) and b).

   We have clarified the assumptions in the revised version. The original version did not distinguish clearly between assumptions that were necessary to allow study of the main effect, and assumptions that were included to present a full model but that could have been chosen otherwise without affecting the results.

   This has now been made much clearer. Regarding the noise issue (point 1), we have clarified the main strategy pursued by the model namely decorrelation, we acknowledge other possible strategies, and we make clear whether and how noise could be incorporated into the model. Regarding the biological plausibility of our assumptions (point 2),

   Reviewer #2 (Public Review):

   The authors propose a new way of looking at the amount of cortical resources (neurons, synapses, and surface area) allocated to process information coming from multiple sensory areas. This is the first theoretical treatment of attempting to answer this question with the framework of efficient coding that states that information should be preserved as much as possible throughout the early sensory stages. This is especially important when there is an explicit bottleneck such that some information has to be discarded. In this current paper, the bottleneck is quantified as the number of dimensions in a continuous space. Using only the second-order statistics of the stimulus, and assuming only the second-order statistics carrying information, the authors use variance instead of Shannon's information. The result is a non-trivial analysis of ordering in the eigenvalues of the corresponding representations. Using clever mathematical approximations, the authors arrive at an analytical expression -- advantageous since numerical evaluation of this problem is tricky due to the long thin tails of the eigenvalues of the chosen covariance function (common in decaying translation-invariant covariances). By changing the relative stimulus power (activity ratio), receptor density (effectively the width of the covariance function), and the truncation of dimensions (bottleneck width), they show that the cortical allocation ratio, surprisingly, is a non-trivial function of such variables. There are a number of weaknesses in this approach, however, it produced valuable insights that have a potential to start a new field of studying such resource allocation problems all across different sensory systems in different animals.

   Strengths

   • A new application of the efficient coding framework to a neural resource allocation problem given acommon bottleneck for multiple independent input regions. It's an innovation (initial results presented at NeurIPS 2019) that brings normative theory with qualitative predictions that may shed new light to seemingly disproportionate cortical allocations. This problem did not have a normative treatment prior to this paper.

   • New insights into allocation of encoding resources as a function of bottleneck, stimulus distribution, andreceptor density. The cortical allocation ratios have nontrivial relations that were not shown before.

   • An analytical method for approximating ordered eigenvalues for a specific stimulus distribution.

   Weaknesses

   The analysis is limited to noiseless systems. This may be a good approximation in the high signal-to-noise ratio regime. However, since the analysis of allocation ratio is very sensitive to the tail of eigenvalue distribution (and their relative rank order), not all conclusions from the current analysis may be robust. Supplemental figure S5 perhaps paints a better picture since it defines the bottleneck as a function of total variance explained instead of number of dimensions. The non-monotonic nonlinear effects are indeed mostly in the last 10% or so of the total variance.

   We agree that the model is most likely to apply in the low-noise regime, as stated in the Discussion. The robustness of the results is indeed a worry, and indeed we have encountered some difficulties when calculating model results numerically due to the issue pointed out by the reviewer, and this led us to focus on an analytical approach in the first case. However, to test model robustness we have now included numerical results for several other covariance functions to demonstrate that, at least qualitatively, the results presented in the paper are not simply a consequence of the particular correlation structure we investigated.

   In case where the stimulus distribution is Guassian, the proposed covariance implies that the stimulus distribution is limited to spatial Gaussian processes with Ornstein-Uhlenbeck prior with two parameters: (inverse) length-scale and variance. While this special case allowed the authors to approach the problem analytically, it is not a widely used natural stimuli distribution as far as I know. This assumed covariance in the stimulus space is quite rough, i.e., each realization of the stimulus is spatially continuous isn't differentiable. In terms of texture, this corresponds to rough surfaces. Of course, if the stimulus distribution is not Gaussian, this may not be the case. However, the authors only described the distribution in terms of the covariance function, and lacks additional detail to fill in this gap.

   We would argue that somewhat ‘rough’ covariance structure might be relatively common, for example in vision objects have clear borders leading to a power law relation and similarly in touch objects are either in contact with the skin or they are not. In either case, we have now extended the analysis to test several other covariance functions numerically. We found that, qualitatively, the main effects described in the paper were still present, though they could differ quantitatively. Interestingly, the convergence limit appeared to depend on the roughness/smoothness of the covariance function, indicating that this might be an important factor.

   The neural response model is unrealistic: Neuronal responses are assumed to be continuous with arbitrary variance. Since the signal is carried by the variance in this manuscript, the resource allocation counts the linear dimensions that this arbitrary variance can be encoded in. Suppose there are 100 neurons that encode a single external variable, for example, a uniform pressure plate stimulus that matches the full range of each sensory receptor. For this stimulus statistics, the variance of all neurons can be combined to a single cortical neuron with 100 times the variance of a single receptor neuron. In this contrived example, the problem is that the cortical neuron can't physiologically have 100 times the variance of the sensory neuron. This study is lacking power constraint that most efficient coding frameworks have (e.g. Atick & Redlich 1990).

   We agree that the response model, as presented, is very simplistic. However, the model can easily be extended to include a variety of constraints, including power constraints, without affecting the results at all. Unfortunately, we did not make this clear enough in the original version. The underlying reason is that decorrelation does not uniquely specify a linear transform and the remaining degrees of freedom can be used to enforce other constraints. As the allocation depends only on the decorrelation process (via PCA), we do not explicitly calculate receptive fields in the paper and any additional constraints (power, sparsity) would affect the receptive fields only and so were left out in the original specification. We have now added clearer pointers for how these could be included and why their inclusion would not affect the present results.

   The star-nosed mole shows that the usage statistics (translated to activity ratio) better explains the cortical allocation than the receptor density. However, the evidence presented for the full model being better than either factor is weak.

   We agree that the results do not present definitive evidence that the model directly accounts for cortical allocations and as we state in the paper, much stronger tests would be needed. Our idea here was to test whether, in principle, the model predictions are compatible with empirical evidence and therefore whether such models could become plausible candidates for explaining neural resource allocation problems. This seems to be the case, even though the evidence in favour of the ‘full model’ versus the ‘activity only’ model is indeed not overwhelming (though this might be expected as the regional differences in activity levels are much greater than those in density). We have now added additional tests to show that the results are not trivial. We would also like to note that it is not obvious that the ‘full’ model would perform better than the ‘activity only’ model: for either we choose the best-fitting bottleneck width (as the true bottleneck width is unknown), and therefore the degrees of freedom are equal (with both activity levels and densities fixed by empirical data).

   Reviewer #3 (Public Review):

   This work follows on a large body of work on efficient coding in sensory processing, but adds a novel angle: How do non-uniform receptor densities and non-uniform stimulus statistics affect the optimal sensory representation?

   The authors start with the motivating example of fingers and tactile receptors, which is well chosen, as it is not overstudied in the efficient coding literature. However, the connection between their model and the example seems to break down after a few lines when the authors state that they treat individual regions as independent, and set the covariance terms to zero. For finger, e.g. that would seem highly implausible, because we typically grasp objects with more than one finger, so that they will be frequently coactivated.

   Our aim was to take a first stab at a model that could theoretically account for neural resource allocation under changes in receptor density and activity levels, and by necessity this initial model is rather simple. Choosing a monotonically decreasing covariance function along with some other simplifications allowed us to quantify the most basic effects, and do so analytically. Any future work should take more complex scenarios into account. Regarding the sense of touch, we agree that the correlational structure of the receptor inputs will be more complex than assumed here, however, whether and how this would affect the results is less clear: Across all tactile experiences (not just grasps, but also single finger activities like typing), cross-finger correlations might not be large compared to intra-finger ones. Unfortunately, there is currently relatively little empirical data on this. That said, we agree with the broader point that complex correlational structure can be found in sensory systems and would need to be taken into account when efficiently representing this information.

   The bottleneck model posited by the authors requires global connectivity as they implement the bottleneck simply by limiting the number of eigenvectors that are used. Thus, in their model, every receptor potentially needs to be connected with every bottleneck neuron. One could also imagine more localized connectivity schemes that would seem more physiologically plausible given the observed connectivity patterns between receptors and relay neurons (e.g. in LGN in the visual system). It would be very interesting to know how this affects the predictions of the theory.

   We agree that the model in its current form is not biologically plausible. While individual receptive fields can be extremely localised, the initial allocation of neurons to regions we describe in the paper relies on a global PCA, and it is not clear how this might be arrived at in practice under biological constraints. However, our aim here was to specify a normative model that generates the optimal allocation and thereby answer what the brain should be doing under ideal circumstances. Future work should definitely ask whether and how these allocations might be worked out in practice and how biological constraints would affect the solutions.

   The representation of the results in the figures is very dense and due to the complex interplay between various factors not easy to digest. This paper would benefit tremendously from an interactive component, where parameters of the model can be changed, and the resulting surfaces and curves are updated.

   We have aimed to make the figures as clear as possible, but do appreciate that the results are relatively complex as they depend on multiple parameters. The code for re-creating the figures is available on Github (https://github.com/lauraredmondson/expansion_contraction_sensory_bottlenecks), making it easy to explore scenarios not described in the paper.

   For parts of the manuscript, not all conclusions made by the authors seem to follow directly from the figures: For example, the authors interpret Fig. 3 as showing that activation ratio determines more strongly whether a sensory representation expands or contracts than density ratio. This is true for small bottlenecks, but for relatively generous ones it seems the other way around. The interpretation by the authors, however, fits better the next paragraph, where they argue that the sensory resources should be relatively constant across the lifespan of an animal, and only stimulus statistics adapt. However, there are notable exceptions - for example, in a drastic example zebrafish change their sensory layout of the retina completely between larvae and adult.

   We have amended the text for this section in the paper to more closely reflect the conclusions that can be drawn from the figure. These are summarised below.

   The purpose of Fig. 3B is to show that knowledge of the activation ratio provides more information about the possible regime of the bottleneck allocations. We cannot tell the magnitude of the expansion or contraction from this information alone, or where in the bottleneck the expansion or contraction would occur. Typically, when we know the activation ratio only, we can tell whether regions will be expanded or contracted or whether both occur over all bottleneck sizes. For a given activation ratio (for example, a = 1:2, as shown in the 3B), we know that the lower activation region can be either contracted only or both expanded and contracted over the course of the bottleneck. In this case, regardless of the density ratio, the lower activation region cannot be contracted only. Conversely, for any density ratio (see dashed horizontal line in Fig. 3B), allocations can be in any regime.

   In the final part of the manuscript, the authors apply their framework to the star nosed mole model system, which has some interesting properties; in particular, relevant parameters seem to be known. Fitting to their interpretation of the modeling outcomes, they conclude that a model that only captures stimulus statistics suffices to model the observed cortical allocations. However, additional work is necessary to make this point convincingly.

   We have now included a further supplementary figure panel providing more details on the fitting procedure and results for each model. Given that we fit over a wide range of bottleneck sizes, where allocations for each ray can vary widely (see Figure 6, supplement 1A), we tested an additional model to confirm that the model requires accurate empirical density and/or activation values for each ray to provide a good fit to cortical data. Here we randomise the values for the density and activation of each ray within the possible range of values for each. We find that with this randomisation of the values the model performs poorly on fitting even with a range of bottleneck sizes. This suggests that the model can only be fitted to the empirical cortical data when using the empirically measured values.

   Read the original source
2. 

   Version published to 10.7554/elife.70777 on eLife

   Aug 4, 2022
3. 

   eLife

   Feb 2, 2022

   Author Response:

   Reviewer #1 (Public Review):

   Edmondson et al. develop an efficient coding approach to study resource allocation in resource constrained sensory systems, with a particular focus on somatosensory representations. Their approach is based on a simple, yet novel insight. Namely - to achieve output decorrelation when encoding stimuli from regions with different input statistics, neurons in the sensory bottleneck should be allocated to these regions according to jointly sorted eigenvalues of the input covariance matrix. The authors demonstrate that, even in a simple scenario, this allocation scheme leads to a complex, non-monotonic relationship between the number of neurons representing each region, receptor density and input statistics. To demonstrate the utility of their approach, the authors generate predictions about …

   Author Response:

   Reviewer #1 (Public Review):

   Edmondson et al. develop an efficient coding approach to study resource allocation in resource constrained sensory systems, with a particular focus on somatosensory representations. Their approach is based on a simple, yet novel insight. Namely - to achieve output decorrelation when encoding stimuli from regions with different input statistics, neurons in the sensory bottleneck should be allocated to these regions according to jointly sorted eigenvalues of the input covariance matrix. The authors demonstrate that, even in a simple scenario, this allocation scheme leads to a complex, non-monotonic relationship between the number of neurons representing each region, receptor density and input statistics. To demonstrate the utility of their approach, the authors generate predictions about cortical representations in the star-nosed mole, and observe a close match between theory and data.

   Strengths:

   These results are certainly interesting and address an issue which to my knowledge has not been studied in-depth before. Touch is a sensory modality rarely mentioned in theoretical studies of sensory coding, and this work contributes to this direction of research.

   A clear strength of the paper is that it demonstrates the existence of non-trivial dependence between resource allocation, bottleneck size and input statistics. Discussion of this relationship highlights the importance of nuance and subtlety in theoretical predictions in neuroscience.

   The proposed theory can be applied to interpret experimental observations - as demonstrated with the example of the star-nosed mole. The prediction of cortical resource allocation is a close match to experimental data.

   We thank the reviewer for the feedback. Indeed, demonstrating an ‘interesting’ effect in even such a simple model was one of the main aims.

   Weaknesses:

   The central weakness of this work are the strong assumptions which are not clearly stated. In result, the consequences of these assumptions are not discussed in sufficient depth which may limit the generality of the proposed approach. In particular:

   1.The paper focuses on a setting with vanishing input noise, where the efficient coding strategy is to reduce the redundancy of the output (for example through decorrelation). This is fine, however, it is not a general efficient coding solution as indicated in the introduction - it is a specific scenario with concrete assumptions, which should be clearly discussed from the beginning.

   2.The model assumes that the goal of the system is to generate outputs, whose covariance structure is an identity matrix (Eq. 1). This corresponds to three assumptions: a) variances of output neurons are equalized, b) the total amount of output variance is equal to M (i.e. the number of of output neurons), c) the activity of output neurons is decorrelated. The paper focuses only on the assumption c), and does not discuss consequences or biological plausibility of assumptions a) and b).

   We have clarified the assumptions in the revised version. The original version did not distinguish clearly between assumptions that were necessary to allow study of the main effect, and assumptions that were included to present a full model but that could have been chosen otherwise without affecting the results. This has now been made much clearer. Regarding the noise issue (point 1), we have clarified the main strategy pursued by the model namely decorrelation, we acknowledge other possible strategies, and we make clear whether and how noise could be incorporated into the model. Regarding the biological plausibility of our assumptions (point 2).

   Reviewer #2 (Public Review):

   The authors propose a new way of looking at the amount of cortical resources (neurons, synapses, and surface area) allocated to process information coming from multiple sensory areas. This is the first theoretical treatment of attempting to answer this question with the framework of efficient coding that states that information should be preserved as much as possible throughout the early sensory stages. This is especially important when there is an explicit bottleneck such that some information has to be discarded. In this current paper, the bottleneck is quantified as the number of dimensions in a continuous space. Using only the second-order statistics of the stimulus, and assuming only the second-order statistics carrying information, the authors use variance instead of Shannon's information. The result is a non-trivial analysis of ordering in the eigenvalues of the corresponding representations. Using clever mathematical approximations, the authors arrive at an analytical expression -- advantageous since numerical evaluation of this problem is tricky due to the long thin tails of the eigenvalues of the chosen covariance function (common in decaying translation-invariant covariances). By changing the relative stimulus power (activity ratio), receptor density (effectively the width of the covariance function), and the truncation of dimensions (bottleneck width), they show that the cortical allocation ratio, surprisingly, is a non-trivial function of such variables. There are a number of weaknesses in this approach, however, it produced valuable insights that have a potential to start a new field of studying such resource allocation problems all across different sensory systems in different animals.

   ##Strengths

   *A new application of the efficient coding framework to a neural resource allocation problem given a common bottleneck for multiple independent input regions. It's an innovation (initial results presented at NeurIPS 2019) that brings normative theory with qualitative predictions that may shed new light to seemingly disproportionate cortical allocations. This problem did not have a normative treatment prior to this paper.

   *New insights into allocation of encoding resources as a function of bottleneck, stimulus distribution, and receptor density. The cortical allocation ratios have nontrivial relations that were not shown before.

   *An analytical method for approximating ordered eigenvalues for a specific stimulus distribution.

   ##Weaknesses

   The analysis is limited to noiseless systems. This may be a good approximation in the high signal-to-noise ratio regime. However, since the analysis of allocation ratio is very sensitive to the tail of eigenvalue distribution (and their relative rank order), not all conclusions from the current analysis may be robust. Supplemental figure S5 perhaps paints a better picture since it defines the bottleneck as a function of total variance explained instead of number of dimensions. The non-monotonic nonlinear effects are indeed mostly in the last 10% or so of the total variance.

   We agree that the model is most likely to apply in the low-noise regime, as stated in the Discussion. The robustness of the results is indeed a worry, and indeed we have encountered some difficulties when calculating model results numerically due to the issue pointed out by the reviewer, and this led us to focus on an analytical approach in the first case. However, to test model robustness we have now included numerical results for several other covariance functions to demonstrate that, at least qualitatively, the results presented in the paper are not simply a consequence of the particular correlation structure we investigated.

   In case where the stimulus distribution is Guassian, the proposed covariance implies that the stimulus distribution is limited to spatial Gaussian processes with Ornstein-Uhlenbeck prior with two parameters: (inverse) length-scale and variance. While this special case allowed the authors to approach the problem analytically, it is not a widely used natural stimuli distribution as far as I know. This assumed covariance in the stimulus space is quite rough, i.e., each realization of the stimulus is spatially continuous isn't differentiable. In terms of texture, this corresponds to rough surfaces. Of course, if the stimulus distribution is not Gaussian, this may not be the case. However, the authors only described the distribution in terms of the covariance function, and lacks additional detail to fill in this gap.

   We would argue that somewhat ‘rough’ covariance structure might be relatively common, for example in vision objects have clear borders leading to a power law relation and similarly in touch objects are either in contact with the skin or they are not. In either case, we have now extended the analysis to test several other covariance functions numerically. We found that, qualitatively, the main effects described in the paper were still present, though they could differ quantitatively. Interestingly, the convergence limit appeared to depend on the roughness/smoothness of the covariance function, indicating that this might be an important factor.

   The neural response model is unrealistic: Neuronal responses are assumed to be continuous with arbitrary variance. Since the signal is carried by the variance in this manuscript, the resource allocation counts the linear dimensions that this arbitrary variance can be encoded in. Suppose there are 100 neurons that encode a single external variable, for example, a uniform pressure plate stimulus that matches the full range of each sensory receptor. For this stimulus statistics, the variance of all neurons can be combined to a single cortical neuron with 100 times the variance of a single receptor neuron. In this contrived example, the problem is that the cortical neuron can't physiologically have 100 times the variance of the sensory neuron. This study is lacking power constraint that most efficient coding frameworks have (e.g. Atick & Redlich 1990).

   We agree that the response model, as presented, is very simplistic. However, the model can easily be extended to include a variety of constraints, including power constraints, without affecting the results at all. Unfortunately, we did not make this clear enough in the original version. The underlying reason is that decorrelation does not uniquely specify a linear transform and the remaining degrees of freedom can be used to enforce other constraints. As the allocation depends only on the decorrelation process (via PCA), we do not explicitly calculate receptive fields in the paper and any additional constraints (power, sparsity) would affect the receptive fields only and so were left out in the original specification. We have now added clearer pointers for how these could be included and why their inclusion would not affect the present results.

   The star-nosed mole shows that the usage statistics (translated to activity ratio) better explains the cortical allocation than the receptor density. However, the evidence presented for the full model being better than either factor is weak.

   We agree that the results do not present definitive evidence that the model directly accounts for cortical allocations and as we state in the paper, much stronger tests would be needed. Our idea here was to test whether, in principle, the model predictions are compatible with empirical evidence and therefore whether such models could become plausible candidates for explaining neural resource allocation problems. This seems to be the case, even though the evidence in favour of the ‘full model’ versus the ‘activity only’ model is indeed not overwhelming (though this might be expected as the regional differences in activity levels are much greater than those in density). We have now added additional tests to show that the results are not trivial. We would also like to note that it is not obvious that the ‘full’ model would perform better than the ‘activity only’ model: for either we choose the best-fitting bottleneck width (as the true bottleneck width is unknown), and therefore the degrees of freedom are equal (with both activity levels and densities fixed by empirical data).

   Reviewer #3 (Public Review):

   This work follows on a large body of work on efficient coding in sensory processing, but adds a novel angle: How do non-uniform receptor densities and non-uniform stimulus statistics affect the optimal sensory representation? The authors start with the motivating example of fingers and tactile receptors, which is well chosen, as it is not overstudied in the efficient coding literature. However, the connection between their model and the example seems to break down after a few lines when the authors state that they treat individual regions as independent, and set the covariance terms to zero. For finger, e.g. that would seem highly implausible, because we typically grasp objects with more than one finger, so that they will be frequently coactivated.

   Our aim was to take a first stab at a model that could theoretically account for neural resource allocation under changes in receptor density and activity levels, and by necessity this initial model is rather simple. Choosing a monotonically decreasing covariance function along with some other simplifications allowed us to quantify the most basic effects, and do so analytically. Any future work should take more complex scenarios into account. Regarding the sense of touch, we agree that the correlational structure of the receptor inputs will be more complex than assumed here, however, whether and how this would affect the results is less clear: Across all tactile experiences (not just grasps, but also single finger activities like typing), cross-finger correlations might not be large compared to intra-finger ones. Unfortunately, there is currently relatively little empirical data on this. That said, we agree with the broader point that complex correlational structure can be found in sensory systems and would need to be taken into account when efficiently representing this information.

   The bottleneck model posited by the authors requires global connectivity as they implement the bottleneck simply by limiting the number of eigenvectors that are used. Thus, in their model, every receptor potentially needs to be connected with every bottleneck neuron. One could also imagine more localized connectivity schemes that would seem more physiologically plausible given the observed connectivity patterns between receptors and relay neurons (e.g. in LGN in the visual system). It would be very interesting to know how this affects the predictions of the theory.

   We agree that the model in its current form is not biologically plausible. While individual receptive fields can be extremely localised, the initial allocation of neurons to regions we describe in the paper relies on a global PCA, and it is not clear how this might be arrived at in practice under biological constraints. However, our aim here was to specify a normative model that generates the optimal allocation and thereby answer what the brain should be doing under ideal circumstances. Future work should definitely ask whether and how these allocations might be worked out in practice and how biological constraints would affect the solutions.

   The representation of the results in the figures is very dense and due to the complex interplay between various factors not easy to digest. This paper would benefit tremendously from an interactive component, where parameters of the model can be changed, and the resulting surfaces and curves are updated.

   We have aimed to make the figures as clear as possible, but do appreciate that the results are relatively complex as they depend on multiple parameters. The code for re-creating the figures is available on Github (https://github.com/lauraredmondson/expansion_contraction_sensory_bottlenecks), making it easy to explore scenarios not described in the paper.

   For parts of the manuscript, not all conclusions made by the authors seem to follow directly from the figures: For example, the authors interpret Fig. 3 as showing that activation ratio determines more strongly whether a sensory representation expands or contracts than density ratio. This is true for small bottlenecks, but for relatively generous ones it seems the other way around. The interpretation by the authors, however, fits better the next paragraph, where they argue that the sensory resources should be relatively constant across the lifespan of an animal, and only stimulus statistics adapt. However, there are notable exceptions - for example, in a drastic example zebrafish change their sensory layout of the retina completely between larvae and adult.

   We have amended the text for this section in the paper to more closely reflect the conclusions that can be drawn from the figure. These are summarised below. The purpose of Fig. 3B is to show that knowledge of the activation ratio provides more information about the possible regime of the bottleneck allocations. We cannot tell the magnitude of the expansion or contraction from this information alone, or where in the bottleneck the expansion or contraction would occur. Typically, when we know the activation ratio only, we can tell whether regions will be expanded or contracted or whether both occur over all bottleneck sizes. For a given activation ratio (for example, a = 1:2, as shown in the 3B), we know that the lower activation region can be either contracted only or both expanded and contracted over the course of the bottleneck. In this case, regardless of the density ratio, the lower activation region cannot be contracted only. Conversely, for any density ratio (see dashed horizontal line in Fig. 3B), allocations can be in any regime.

   In the final part of the manuscript, the authors apply their framework to the star nosed mole model system, which has some interesting properties; in particular, relevant parameters seem to be known. Fitting to their interpretation of the modeling outcomes, they conclude that a model that only captures stimulus statistics suffices to model the observed cortical allocations. However, additional work is necessary to make this point convincingly.

   We have now included a further supplementary figure panel providing more details on the fitting procedure and results for each model. Given that we fit over a wide range of bottleneck sizes, where allocations for each ray can vary widely (see Figure 6, supplement 1A), we tested an additional model to confirm that the model requires accurate empirical density and/or activation values for each ray to provide a good fit to cortical data. Here we randomise the values for the density and activation of each ray within the possible range of values for each. We find that with this randomisation of the values the model performs poorly on fitting even with a range of bottleneck sizes. This suggests that the model can only be fitted to the empirical cortical data when using the empirically measured values.

   Read the original source
4. 

   eLife

   Feb 2, 2022

   Evaluation Summary:

   The paper develops a mathematical approach to study the allocation of cortical area to sensory representations in the presence of resource constraints. The theory is applied to study sensory representations in the somatosensory system. This problem is largely unexplored, the results are novel and can be of interest to experimental and theoretical neuroscientists.

   (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #2 agreed to share their name with the authors.)

   Read the original source
5. 

   eLife

   Feb 2, 2022

   Reviewer #1 (Public Review):

   Edmondson et al. develop an efficient coding approach to study resource allocation in resource constrained sensory systems, with a particular focus on somatosensory representations. Their approach is based on a simple, yet novel insight. Namely - to achieve output decorrelation when encoding stimuli from regions with different input statistics, neurons in the sensory bottleneck should be allocated to these regions according to jointly sorted eigenvalues of the input covariance matrix. The authors demonstrate that, even in a simple scenario, this allocation scheme leads to a complex, non-monotonic relationship between the number of neurons representing each region, receptor density and input statistics. To demonstrate the utility of their approach, the authors generate predictions about cortical …

   Reviewer #1 (Public Review):

   Edmondson et al. develop an efficient coding approach to study resource allocation in resource constrained sensory systems, with a particular focus on somatosensory representations. Their approach is based on a simple, yet novel insight. Namely - to achieve output decorrelation when encoding stimuli from regions with different input statistics, neurons in the sensory bottleneck should be allocated to these regions according to jointly sorted eigenvalues of the input covariance matrix. The authors demonstrate that, even in a simple scenario, this allocation scheme leads to a complex, non-monotonic relationship between the number of neurons representing each region, receptor density and input statistics. To demonstrate the utility of their approach, the authors generate predictions about cortical representations in the star-nosed mole, and observe a close match between theory and data.

   Strengths:

   These results are certainly interesting and address an issue which to my knowledge has not been studied in-depth before. Touch is a sensory modality rarely mentioned in theoretical studies of sensory coding, and this work contributes to this direction of research.

   A clear strength of the paper is that it demonstrates the existence of non-trivial dependence between resource allocation, bottleneck size and input statistics. Discussion of this relationship highlights the importance of nuance and subtlety in theoretical predictions in neuroscience.

   The proposed theory can be applied to interpret experimental observations - as demonstrated with the example of the star-nosed mole. The prediction of cortical resource allocation is a close match to experimental data.

   Weaknesses:

   The central weakness of this work are the strong assumptions which are not clearly stated. In result, the consequences of these assumptions are not discussed in sufficient depth which may limit the generality of the proposed approach. In particular:

   1. The paper focuses on a setting with vanishing input noise, where the efficient coding strategy is to reduce the redundancy of the output (for example through decorrelation). This is fine, however, it is not a general efficient coding solution as indicated in the introduction - it is a specific scenario with concrete assumptions, which should be clearly discussed from the beginning.

   2. The model assumes that the goal of the system is to generate outputs, whose covariance structure is an identity matrix (Eq. 1). This corresponds to three assumptions: a) variances of output neurons are equalized, b) the total amount of output variance is equal to M (i.e. the number of of output neurons), c) the activity of output neurons is decorrelated. The paper focuses only on the assumption c), and does not discuss consequences or biological plausibility of assumptions a) and b).

   Read the original source
6. 

   eLife

   Feb 2, 2022

   Reviewer #2 (Public Review):

   The authors propose a new way of looking at the amount of cortical resources (neurons, synapses, and surface area) allocated to process information coming from *multiple* sensory areas. This is the first theoretical treatment of attempting to answer this question with the framework of efficient coding that states that information should be preserved as much as possible throughout the early sensory stages. This is especially important when there is an explicit bottleneck such that some information has to be discarded. In this current paper, the bottleneck is quantified as the number of dimensions in a continuous space. Using only the second-order statistics of the stimulus, and assuming only the second-order statistics carrying information, the authors use variance instead of Shannon's information. The result …

   Reviewer #2 (Public Review):

   The authors propose a new way of looking at the amount of cortical resources (neurons, synapses, and surface area) allocated to process information coming from *multiple* sensory areas. This is the first theoretical treatment of attempting to answer this question with the framework of efficient coding that states that information should be preserved as much as possible throughout the early sensory stages. This is especially important when there is an explicit bottleneck such that some information has to be discarded. In this current paper, the bottleneck is quantified as the number of dimensions in a continuous space. Using only the second-order statistics of the stimulus, and assuming only the second-order statistics carrying information, the authors use variance instead of Shannon's information. The result is a non-trivial analysis of ordering in the eigenvalues of the corresponding representations. Using clever mathematical approximations, the authors arrive at an analytical expression -- advantageous since numerical evaluation of this problem is tricky due to the long thin tails of the eigenvalues of the chosen covariance function (common in decaying translation-invariant covariances). By changing the relative stimulus power (activity ratio), receptor density (effectively the width of the covariance function), and the truncation of dimensions (bottleneck width), they show that the cortical allocation ratio, surprisingly, is a non-trivial function of such variables. There are a number of weaknesses in this approach, however, it produced valuable insights that have a potential to start a new field of studying such resource allocation problems all across different sensory systems in different animals.

   ## Strengths

   * A new application of the efficient coding framework to a neural resource allocation problem given a common bottleneck for multiple independent input regions. It's an innovation (initial results presented at NeurIPS 2019) that brings normative theory with qualitative predictions that may shed new light to seemingly disproportionate cortical allocations. This problem did not have a normative treatment prior to this paper.

   * New insights into allocation of encoding resources as a function of bottleneck, stimulus distribution, and receptor density. The cortical allocation ratios have nontrivial relations that were not shown before.

   * An analytical method for approximating ordered eigenvalues for a specific stimulus distribution.

   ## Weaknesses

   The analysis is limited to noiseless systems. This may be a good approximation in the high signal-to-noise ratio regime. However, since the analysis of allocation *ratio* is very sensitive to the tail of eigenvalue distribution (and their relative rank order), not all conclusions from the current analysis may be **robust**. Supplemental figure S5 perhaps paints a better picture since it defines the bottleneck as a function of total variance explained instead of number of dimensions. The non-monotonic nonlinear effects are indeed mostly in the last 10% or so of the total variance.

   In case where the stimulus distribution is Guassian, the proposed covariance implies that the stimulus distribution is limited to spatial Gaussian processes with Ornstein-Uhlenbeck prior with two parameters: (inverse) length-scale and variance. While this special case allowed the authors to approach the problem analytically, it is not a widely used natural stimuli distribution as far as I know. This assumed covariance in the stimulus space is quite *rough*, i.e., each realization of the stimulus is spatially continuous isn't differentiable. In terms of texture, this corresponds to rough surfaces. Of course, if the stimulus distribution is not Gaussian, this may not be the case. However, the authors only described the distribution in terms of the covariance function, and lacks additional detail to fill in this gap.

   The neural response model is unrealistic: Neuronal responses are assumed to be continuous with arbitrary variance. Since the signal is carried by the variance in this manuscript, the resource allocation counts the linear dimensions that this arbitrary variance can be encoded in. Suppose there are 100 neurons that encode a single external variable, for example, a uniform pressure plate stimulus that matches the full range of each sensory receptor. For this stimulus statistics, the variance of all neurons can be combined to a single cortical neuron with 100 times the variance of a single receptor neuron. In this contrived example, the problem is that the cortical neuron can't physiologically have 100 times the variance of the sensory neuron. This study is lacking power constraint that most efficient coding frameworks have (e.g. Atick & Redlich 1990).

   The star-nosed mole shows that the usage statistics (translated to activity ratio) better explains the cortical allocation than the receptor density. However, the evidence presented for the full model being better than either factor is weak.

   Read the original source
7. 

   eLife

   Feb 2, 2022

   Reviewer #3 (Public Review):

   This work follows on a large body of work on efficient coding in sensory processing, but adds a novel angle: How do non-uniform receptor densities and non-uniform stimulus statistics affect the optimal sensory representation?

   The authors start with the motivating example of fingers and tactile receptors, which is well chosen, as it is not overstudied in the efficient coding literature. However, the connection between their model and the example seems to break down after a few lines when the authors state that they treat individual regions as independent, and set the covariance terms to zero. For finger, e.g. that would seem highly implausible, because we typically grasp objects with more than one finger, so that they will be frequently coactivated.

   The bottleneck model posited by the authors requires global …

   Reviewer #3 (Public Review):

   This work follows on a large body of work on efficient coding in sensory processing, but adds a novel angle: How do non-uniform receptor densities and non-uniform stimulus statistics affect the optimal sensory representation?

   The authors start with the motivating example of fingers and tactile receptors, which is well chosen, as it is not overstudied in the efficient coding literature. However, the connection between their model and the example seems to break down after a few lines when the authors state that they treat individual regions as independent, and set the covariance terms to zero. For finger, e.g. that would seem highly implausible, because we typically grasp objects with more than one finger, so that they will be frequently coactivated.

   The bottleneck model posited by the authors requires global connectivity as they implement the bottleneck simply by limiting the number of eigenvectors that are used. Thus, in their model, every receptor potentially needs to be connected with every bottleneck neuron. One could also imagine more localized connectivity schemes that would seem more physiologically plausible given the observed connectivity patterns between receptors and relay neurons (e.g. in LGN in the visual system). It would be very interesting to know how this affects the predictions of the theory.

   The representation of the results in the figures is very dense and due to the complex interplay between various factors not easy to digest. This paper would benefit tremendously from an interactive component, where parameters of the model can be changed, and the resulting surfaces and curves are updated.

   For parts of the manuscript, not all conclusions made by the authors seem to follow directly from the figures: For example, the authors interpret Fig. 3 as showing that activation ratio determines more strongly whether a sensory representation expands or contracts than density ratio. This is true for small bottlenecks, but for relatively generous ones it seems the other way around. The interpretation by the authors, however, fits better the next paragraph, where they argue that the sensory resources should be relatively constant across the lifespan of an animal, and only stimulus statistics adapt. However, there are notable exceptions - for example, in a drastic example zebrafish change their sensory layout of the retina completely between larvae and adult.

   In the final part of the manuscript, the authors apply their framework to the star nosed mole model system, which has some interesting properties; in particular, relevant parameters seem to be known. Fitting to their interpretation of the modeling outcomes, they conclude that a model that only captures stimulus statistics suffices to model the observed cortical allocations. However, additional work is necessary to make this point convincingly.

   Read the original source
8. 

   Version published to 10.1101/2021.05.26.445857v2 on bioRxiv

   Jan 31, 2022
9. 

   Version published to 10.1101/2021.05.26.445857v1 on bioRxiv

   May 27, 2021