Ultrafast simulation of large-scale neocortical microcircuitry with biophysically realistic neurons
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
This paper demonstrates that artificial neural networks can be used to accurately predict the responses of biologically-detailed neuron models to synaptic inputs, and hence to approximate the behaviour of networks of such neurons. This study potentially opens the door to massively reduced simulation times for biologically-detailed neuronal network simulations without recourse to supercomputers and hence will be of broad interest to computational neuroscientists.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #3 agreed to share their name with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Understanding the activity of the mammalian brain requires an integrative knowledge of circuits at distinct scales, ranging from ion channel gating to circuit connectomics. Computational models are regularly employed to understand how multiple parameters contribute synergistically to circuit behavior. However, traditional models of anatomically and biophysically realistic neurons are computationally demanding, especially when scaled to model local circuits. To overcome this limitation, we trained several artificial neural network (ANN) architectures to model the activity of realistic multicompartmental cortical neurons. We identified an ANN architecture that accurately predicted subthreshold activity and action potential firing. The ANN could correctly generalize to previously unobserved synaptic input, including in models containing nonlinear dendritic properties. When scaled, processing times were orders of magnitude faster compared with traditional approaches, allowing for rapid parameter-space mapping in a circuit model of Rett syndrome. Thus, we present a novel ANN approach allowing for rapid, detailed network experiments using inexpensive and commonly available computational resources.
Article activity feed
-
-
Evaluation Summary:
This paper demonstrates that artificial neural networks can be used to accurately predict the responses of biologically-detailed neuron models to synaptic inputs, and hence to approximate the behaviour of networks of such neurons. This study potentially opens the door to massively reduced simulation times for biologically-detailed neuronal network simulations without recourse to supercomputers and hence will be of broad interest to computational neuroscientists.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #3 agreed to share their name with the authors.)
-
Reviewer #1 (Public Review):
Here the authors set out to use neural networks to simulate neurons, which is an intriguing inversion of scales and approaches. They achieve quite remarkable speedups, drawing on the efficiencies of neural network implementations, especially in GPUs. Overall, I find this to be a potentially very exciting development for neuronal modelling as it will bring quite large models within reach of 'ordinary' researchers who don't have easy access to massive supercomputers. I feel the authors should address a couple of major concerns about the reporting of their software and results.
-
Reviewer #2 (Public Review):
The manuscript by Oláh, Pedersen, and Rowan, "Ultrafast Simulation of Large-Scale Neocortical Microcircuitry with Biophysically Realistic Neurons", demonstrates an ANN architecture that accurately captures the dynamics of neurons while also providing a substantial reduction of computing time relative to the standard method using NEURON simulations. This is important work that, together with recent similar work by others, opens exciting opportunities for detailed and accurate simulations of neurons with much higher speed than has been possible so far. The authors demonstrated an important step forward in enabling accurate and high-speed simulations of neurons taking advantage of modern ANN architectures and computing hardware such as GPUs.
• They started by considering several different ANN architectures and …
Reviewer #2 (Public Review):
The manuscript by Oláh, Pedersen, and Rowan, "Ultrafast Simulation of Large-Scale Neocortical Microcircuitry with Biophysically Realistic Neurons", demonstrates an ANN architecture that accurately captures the dynamics of neurons while also providing a substantial reduction of computing time relative to the standard method using NEURON simulations. This is important work that, together with recent similar work by others, opens exciting opportunities for detailed and accurate simulations of neurons with much higher speed than has been possible so far. The authors demonstrated an important step forward in enabling accurate and high-speed simulations of neurons taking advantage of modern ANN architectures and computing hardware such as GPUs.
• They started by considering several different ANN architectures and training each architecture to capture the membrane voltage dynamics in a single-compartment NEURON model of a neuron described by Hodgkin-Huxley equations. One particular architecture, the "CNN-LSTM" was substantially more successful than others in training and testing, especially in reproducing the action potentials (APs), i.e., it captured well both the subthreshold membrane voltage and APs.
• They then showed that once trained on a neuron model, the CNN-LSTM ANN generalizes to the cases where ionic conductance is manipulated and can capture the ionic sodium and potassium currents, in addition to the membrane voltage, without the need to re-train the ANN.
• Furthermore, the ANN generalizes to the cases involving nonlinear synapses, such as NMDA synapses, in addition to the simpler AMPA synapses.
• The authors then applied the ANN simulations to a realistic multi-compartmental model of a pyramidal cell (PC) from Layer 5 (L5PC) in the mammalian cortex and showed good performance for this cell, as well as L2/3PC, L4PC, and L6PC.
• In the next step, between one and 5,000 L5 PCs were simulated, and it was shown that ANN simulations on a GPU are faster by a factor of over 10,000 than NEURON simulations on a single CPU. This is a drastic and impressive difference.
• Finally, the authors demonstrated that a network of 150 PCs mimicking the effects of Rett syndrome can be efficiently simulated using the ANN approach. They sampled a large parameter space using the ANN simulations on a single GPU within seconds, a task that would take again on the order of 10,000 times longer for regular CPU simulations without the ANN approach.
The study is interesting and extensive. Multiple lines of evidence, as enumerated above, show that the approach described here is promising. It definitely deserves the attention of fellow computational neuroscientists. I am convinced that many of them would like to try out the approach and the code that will be shared freely by the authors. The paper is mostly clear and easy to follow.
In my opinion, the weaknesses of this work are relatively minor. Generally, I would like to see more characterization of ANN generalization for tasks relevant to the everyday practice of neuronal modeling. For example, it is common to change not only the strength, but also the number of synapses, and it will be useful if the ANN approach seamlessly generalizes to that. Another point is that the authors could put their study more into the context of other research happening in this area and discuss the uniqueness of their contributions in that light. But, again, overall I find this study to be extensive and interesting, and I think it will generate a strong interest in the field.
-
Reviewer #3 (Public Review):
In this study, the authors' goal was to accelerate biologically-detailed neuronal network simulations, by leveraging the computational efficiency and amenability to parallelization of modern artificial neural network architectures (ANNs). The general idea is to train an ANN on time series generated by traditional neuronal simulations, then provide new inputs and determine whether the generalization capabilities of the ANN allow it to predict the time series that would be generated by the original model with the same inputs.
Starting with a simple, single-compartment neuron model, the authors trained five different ANNs to reproduce/approximate the behaviour of the model and found that only one of these ANNs, a recurrent model containing convolutional layers, long short-term memory layers, and fully connected …
Reviewer #3 (Public Review):
In this study, the authors' goal was to accelerate biologically-detailed neuronal network simulations, by leveraging the computational efficiency and amenability to parallelization of modern artificial neural network architectures (ANNs). The general idea is to train an ANN on time series generated by traditional neuronal simulations, then provide new inputs and determine whether the generalization capabilities of the ANN allow it to predict the time series that would be generated by the original model with the same inputs.
Starting with a simple, single-compartment neuron model, the authors trained five different ANNs to reproduce/approximate the behaviour of the model and found that only one of these ANNs, a recurrent model containing convolutional layers, long short-term memory layers, and fully connected layers, which they termed the CNN-LSTM network, was able to satisfactorily reproduce both the subthreshold activity and the spiking activity (91% of spikes predicted correctly).
The authors then added complexity to the model and found that it generalized well to changes in synaptic weight, non-linear synapses, and changes in input patterns. Partial retraining of the upper layers allowed generalization to other changes (steady-state activation of K channels).
Moving from single-compartment to multi-compartment neurons, the CNN-LSTM network was still able to accurately reproduce sub-threshold fluctuations, although the accuracy of spike prediction declined to 67%.Applying the technique to circuit models, the authors performed a parameter-space mapping in a model of Rett syndrome. In all cases beyond single point-neuron models, the CNN-LSTM network running on a GPU system outperformed the NEURON simulator running on a single CPU. In the case of simulating many identical neurons (but with different inputs), and when simulating the small circuit model of Rett syndrome, the performance gain was of several orders of magnitude.
Strengths:
Although ANNs have been used extensively in approximating the dynamics of neural systems, this has typically been at a much higher level of abstraction and a much coarser anatomical scale. I am not aware of any previous demonstration of using ANNs as a practical tool to approximate the behaviour of individual biological neurons and networks of such neurons (as opposed to using spiking neural networks to perform machine learning tasks, where there is a large literature). This manuscript demonstrates convincingly that this approach is a promising one, and provides a practical starting point for further research on this technique. The comparisons at the single neuron level are particularly thorough and well done, and (i) demonstrate that the trained network displays good generalization, and (ii) provide good evidence that the network has really learned important features of the underlying system.Weaknesses:
The comparisons at the network level are less convincing than for single neurons.
Although this is not stated clearly, it seems that the simulations with 50 and 5000 neurons were for identical neurons differing only in their inputs. Given the variability of neuronal properties even within a specific cell type and the increasing representation of this variability in computational neuroscience models, it would have been valuable to know what the performance impact of incorporating such variability would be, in both the training and exploitation phases.
Furthermore, much less detailed comparisons between the NEURON simulation results and the ANN results are given for the network models. For the Rett syndrome model, no results from the NEURON simulations are shown, neither at the single neuron level nor at the level of parameter maps, which makes it impossible to determine whether the ANN model is adequately reproducing the correct behaviour.
For very large-scale simulations, spike communication is often the rate-limiting factor in simulations, rather than solving the equations for neuronal dynamics. Cortical pyramidal neurons typically receive around 10000 synapses, a much higher number than appears to have been used here. It remains to be demonstrated whether the advantage in computational speed is still an important factor in systems with very high rates of synaptic events, in particular for models that are too large to fit on a single GPU.
It could be argued that the playing field was tilted towards the ANN, since the NEURON simulator cannot benefit from GPU parallelization at the present time. Using a GPU-capable simulator such as GeNN as an additional point of comparison would give a clearer picture of the strengths and weaknesses of the ANN approach.
The Discussion section does not adequately discuss the limitations of the current study, nor does it sufficiently address the potential weaknesses of the ANN approach, although some potential limitations are mentioned.In summary, the authors have shown that ANNs are a promising tool for greatly increasing the scope of what can be modelled with generally available computing hardware, reducing the bottleneck of supercomputer availability. Clearly further studies are needed to explore the utility of the approach in a broader range of real-world scenarios, but by providing a good description of the successful CNN-LSTM model, and by providing their source code in a public repository, the authors have provided a strong basis for others to test this approach in their own projects.
This study is likely to have a substantial impact, stimulating further work in (i) understanding the performance-accuracy tradeoffs of this approach in comparison to other GPU-based simulation methods, and (ii) understanding the modelling domain for which this approach gives adequate accuracy (does it break down for networks on the edge of chaos, for example?). Should the method live up to its promise, it could greatly accelerate progress in computational neuroscience.
-
-