Nonlinear transient amplification in recurrent neural networks with short-term plasticity

  1. Yue Kris Wu
  2. Friedemann Zenke

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    Networks of excitatory neurons in the mammalian cortex are capable of responding rapidly and selectively to incoming stimuli. This rapid response is believed to be due to positive feedback among excitatory cells, which necessitates a stabilizing mechanism at circuit and cellular levels. This modelling study shows how short term plasticity at synapses can stabilize the response of a recurrently connected circuit of excitatory and inhibitory cells, whereas intrinsic spike frequency adaptation is unable to stabilise network responses. These findings deepen our understanding of the various mechanisms that can stabilise circuit dynamics and will be of broad interest to neurophysiologists and theoretical neuroscientists.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

To rapidly process information, neural circuits have to amplify specific activity patterns transiently. How the brain performs this nonlinear operation remains elusive. Hebbian assemblies are one possibility whereby strong recurrent excitatory connections boost neuronal activity. However, such Hebbian amplification is often associated with dynamical slowing of network dynamics, non-transient attractor states, and pathological run-away activity. Feedback inhibition can alleviate these effects but typically linearizes responses and reduces amplification gain. Here, we study nonlinear transient amplification (NTA), a plausible alternative mechanism that reconciles strong recurrent excitation with rapid amplification while avoiding the above issues. NTA has two distinct temporal phases. Initially, positive feedback excitation selectively amplifies inputs that exceed a critical threshold. Subsequently, short-term plasticity quenches the run-away dynamics into an inhibition-stabilized network state. By characterizing NTA in supralinear network models, we establish that the resulting onset transients are stimulus selective and well-suited for speedy information processing. Further, we find that excitatory-inhibitory co-tuning widens the parameter regime in which NTA is possible in the absence of persistent activity. In summary, NTA provides a parsimonious explanation for how excitatory-inhibitory co-tuning and short-term plasticity collaborate in recurrent networks to achieve transient amplification.

Article activity feed

  1. Version published to 10.7554/elife.71263 on eLife
  2. eLife

    Evaluation Summary:

    Networks of excitatory neurons in the mammalian cortex are capable of responding rapidly and selectively to incoming stimuli. This rapid response is believed to be due to positive feedback among excitatory cells, which necessitates a stabilizing mechanism at circuit and cellular levels. This modelling study shows how short term plasticity at synapses can stabilize the response of a recurrently connected circuit of excitatory and inhibitory cells, whereas intrinsic spike frequency adaptation is unable to stabilise network responses. These findings deepen our understanding of the various mechanisms that can stabilise circuit dynamics and will be of broad interest to neurophysiologists and theoretical neuroscientists.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

  3. eLife

    Reviewer #1 (Public Review):

    In this manuscript, the authors introduce a new mechanism, named nonlinear transient amplification (NTA), to explain how cortical circuits can amplify specific patterns of activity without dynamical slowing of network dynamics. Combining analytical and numerical investigations of networks with NTA, the authors provide an extensive characterization of how properties such as pattern completion and separation evolve in time and depend on the tuning of inhibitory cells. Moreover, they show that the NTA mechanism applies also to networks of spiking neurons.

    The idea of combining network dynamics effects, such as the increase of effective coupling between cells with input strength, with short term plasticity is novel and can drive the field to explore how known biophysical properties of neurons and synapses shape brain computations. Two-photon stimulation technologies now allow experimentalists to investigate if and how networks of neurons in the brain implement computation such as pattern completion and separation; new theories like the one introduced in this paper are critical to interpret results obtained in these photostimulation experiments.

    Overall, this is a stimulating and timely paper. However, there are important concerns (see list below) that I think should be addressed in order to state that nonlinear transient amplification underlies computations in the brain.

    1. Critical features of the model do not seem compatible with experimental results (1a and 1b). The authors should either explain how their model can be reconciled with these experimental findings, or acknowledge the limitations of their results.
      1a) The network model is characterized by an input dependent dynamical stability (Figure 2) and, for low inputs, operates in the non-inhibition stabilized regime. However, experiments have shown that cortical circuits, at least in the mouse brain, are in the ISN regime already at baseline, i.e. in the absence of sensory stimuli [Sanzeni et al., eLife 2020].
      1b) The model predicts that pattern completion in cortex should be limited to response onset, and should be followed by suppression of cotuned unstimulated cells (Figure 4). Pattern completion in cortex has been recently investigated in optogenetic experiments (e.g. [Carrillo-Reid et al., Science 2016; Marshal et al., Science 2019]); these experiments have shown that stimulation of a subset of cells activates cotuned unstimulated neurons. Contrary to the model prediction, unstimulated neurons remain active throughout the whole stimulation period, which lasts a few seconds, i.e. much longer than the typical time scale of STP.

    2. The description of balanced amplification done in the text should be improved. In its original formulation [Murphy and Miller, Neuron 2009], balanced amplification has been shown to selectively amplify specific patterns without slowing of network dynamics. This amplification emerges in networks with different ensembles of excitatory cells, each of which is characterized by symmetric connectivity between excitatory cells, and cotuned inhibitory cells. Therefore, the authors should frame their work as an alternative mechanism to balanced amplification, rather than a solution to an unsolved conundrum.

    3. The stability mechanism underlying the results of Fig. 2 should be analyzed in more depth. Fig. 2B suggests that the network is stabilized by STP and not by inhibition, since it does not show runaway activity when inhibition is clamped. Despite this fact, the authors refer to the steady state as inhibition stabilized but it is not clear what evidence supports this claim. The authors use the paradoxical effect as a proxy for inhibitory stabilization, but the implication "paradoxical response->ISN" has been proven only in networks without STP and we do not know if it holds in networks with STP. The "ISN index" defined in Eq. (13) gives information about the stability of the excitatory population but, in networks with STP or STF, does not tell us if the network is stabilized by inhibition or by another mechanism (e.g. STP).

    4. Page 7-8. In networks of excitatory and inhibitory neurons without STP (two degrees of freedom), the derivative of the characteristic function F(z) is related to the eigenvalues of the network Jacobian matrix [Kraynyukova and Tchumatchenko, PNAS 2018]; this relation allows to analyze the stability of a network fixed point by studying the slope of F(z). The authors apply this approach in their model models which has three degrees of freedom (e.g. activity of excitatory-inhibitory neurons and STD variable x). To do so, they should prove that the approach of [Kraynyukova and Tchumatchenko, PNAS 2018] can be generalized to their model, e.g. they should show how the derivative of F(z) relates to the three eigenvalues of the network Jacobian matrix and to the network stability.

  4. eLife

    Reviewer #2 (Public Review):

    The authors study fast transient amplification of external inputs in nonlinear recurrent networks of excitatory (E) and inhibitory (I) neurons with different forms neural or synaptic adaptation: spike frequency adaptation (SFA) or short-term synaptic plasticity (STP). They seek conditions under which nonlinear transient amplification (NTA) at the onset of the stimulus is strong, and yet activity reaches a stable steady-state with not too large (biologically plausible) activity. The mechanism for the NTA is strong recurrent excitation (E-E connections), while the re-stabilization of activity after amplification is provided by the adaptive mechanisms (SFA or STP). They find that while SFA is unable to re-stabilize post-onset activity,

    A) STP - either in the form of short-term depression (STD) of E-E connections, or
    short-term facilitation (STF) of E-to-I connections-- is capable of doing so.

    In addition they demonstrate other features of NTA in networks with STP, which they characterize as follows:

    B) NTA requires symmetric recurrent EE

    C) unlike in linear transient amplification, NTA happens only when stimulus strength is above a nonzero threshold

    D) NTA is orders of magnitude stronger than TA in non-normal linear E/I networks

    E) The post-transient steady state is inhibitory stabilized (ISN) and shows the associated paradoxical effect.

    F) Stimulus onset responses (i.e. during NTA) provide better pattern completion and "pattern separation" (or stimulus selectivity) compared to steady state responses.

    G) E/I co-tuning "broadens the parameter regime of NTA".

    While the study provides novel findings and thus has merit, it is problematic that the main findings A, B & C are actually not novel and have been studied in detail before: they were studied and characterized as such in networks with strong recurrent excitation with STD, by Misha Tsodyks' group (main references are Loebel and Tsodyks 2002, and Loebel, Nelken and Tsodyks 2007, which studied rate networks as in the current study, but also Tsodyks, Uziel & Markram 2000 which studied the phenomenon in spiking nets).

    In particular, Loebel et al. 2007 provided a model of Auditory cortex (that captured various empirically observed aspects of fast onset auditory cortical responses) with E and I neurons, with exactly the same mechanism for NTA (A & B): strong symmetric E-E recurrence, with re-stabilization to a low activity state provided by STD. In particular, property C was demostrated (and conditions for it analytically derived in Loebel et al. 2002) and was used (in Loebel et al. 2007) to account for the V-shaped Frequency-Intensity tuning curves observed in auditory cortex.

    Yet none of these past studies are cited in the current study, and the results A-C are presented as novel.
    Properties D and E also possibly/probably hold in Loebel et al. 2007's model too, but that paper did not study/characterise these features.

    So the findings D-G are indeed novel (to my knowledge), as are the extension of A-C to networks with STF instead of STD (as is the negative result that SFA cannot yield restabilization, I believe).

    Thus I think a major rewriting of the manuscript is in order, in which the main focus (in particular the selling points in Abstract/intro/discussion, e.g. lines 74-75) is on these truly novel findings (D-G) instead of on A-C (which of course could/should still be mentioned). Moreover some of these newly characterized features (in particular D & E which are currently only cursorily mentioned, and were not properly quantified, or their parameter dependence was not studied) should be more thoroughly/quantitatively characterized.

  5. Version published to 10.1101/2021.06.09.447718v1 on bioRxiv