A theory and recipe to construct general and biologically plausible integrating continuous attractor neural networks
Curation statements for this article:-
Curated by eLife
eLife Assessment
This valuable study presents a theoretical framework for building continuous attractor networks that integrate with a wide range of topologies, which are of increasing relevance to neuroscientists. While the work offers solid evidence for most claims, the evidence supporting biological plausibility and key claims - such as the existence of a continuum of stable states and robustness across geometries - is currently incomplete and would benefit from further analysis or discussion. The study will be of interest to computational and systems neuroscientists working on neural dynamics and network models of cognition.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Abstract
Across the brain, circuits with continuous attractor dynamics underpin the representation and storage in memory of continuous variables for motor control, navigation, and mental computations. The represented variables have various dimensions and topologies (lines, rings, euclidean planes), and the circuits exhibit continua of fixed points to store these variables, and the ability to use input velocity signals to update and maintain the representation of unobserved variables, effectively integrating the incoming velocity signal. Integration constitutes a general computational strategy that enables variable state estimation when direct observation of the variable is not possible, suggesting that it may play a critical role in other cognitive processes. While some neural network models for integration exist, a comprehensive theory for constructing neural circuits with a given topology and integration capabilities is lacking. Here, we present a theoretically-driven design framework, Manifold Attractor Direct Engineering (MADE), to automatically, analytically, and explicitly construct biologically plausible continuous attractor neural networks with diverse user-specified topologies. We show how these attractor networks can be endowed with accurate integration functionality through biologically realistic circuit mechanisms. MADE networks closely resemble biological circuits where the attractor mechanisms have been characterized. Additionally, MADE offers innovative and minimal circuit models for uncharacterized topologies, enabling a systematic approach to developing and testing mathematical theories related to cognition and computation in the brain.
Article activity feed
-
eLife Assessment
This valuable study presents a theoretical framework for building continuous attractor networks that integrate with a wide range of topologies, which are of increasing relevance to neuroscientists. While the work offers solid evidence for most claims, the evidence supporting biological plausibility and key claims - such as the existence of a continuum of stable states and robustness across geometries - is currently incomplete and would benefit from further analysis or discussion. The study will be of interest to computational and systems neuroscientists working on neural dynamics and network models of cognition.
-
Reviewer #1 (Public review):
This is a theoretical study addressing the problem of constructing integrator networks for which the activity state and integrated variables display non-trivial topologies. Historically, researchers in theoretical neuroscience have focused on models with simple underlying geometries (e.g., circle, torus), for which analytical models could be more easily constructed. How these models can be generalised to complex scenarios is, however, a non-trivial question. This is furthermore a time-sensitive issue, as population recordings from the brain in complex tasks and environments increasingly require the ability to construct such models.
I believe the authors do a good job of explaining the challenges related to this problem. They also propose a class of models that, although not fully general, overcome many of …
Reviewer #1 (Public review):
This is a theoretical study addressing the problem of constructing integrator networks for which the activity state and integrated variables display non-trivial topologies. Historically, researchers in theoretical neuroscience have focused on models with simple underlying geometries (e.g., circle, torus), for which analytical models could be more easily constructed. How these models can be generalised to complex scenarios is, however, a non-trivial question. This is furthermore a time-sensitive issue, as population recordings from the brain in complex tasks and environments increasingly require the ability to construct such models.
I believe the authors do a good job of explaining the challenges related to this problem. They also propose a class of models that, although not fully general, overcome many of these difficulties while appearing solid and well-functioning. This requires some non-trivial mathematics, which is nevertheless conveyed in a reasonably accessible form. The manuscript is well written, and both the methodology and the code are well documented.
That said, I believe the manuscript has two major limitations, which could be addressed in a revision. First, some of the assumptions underlying this class of models are somewhat restrictive but are not sufficiently discussed. Second, although the stated goal of the manuscript is to provide practical recipes for constructing integrator networks, the methods section is not very explicit about the specific steps required for different geometries. I elaborate on these limitations below.
(1) The authors repeatedly describe MADE as a technique for constructing integrators of specified "topologies and geometries." What do they mean by "geometries"? Intuitively, I would associate geometry with properties beyond topology, such as embedding dimensionality or curvature. However, it is unclear to me to what extent these aspects are explicitly specified or controlled in MADE. It seems that geometry is only indirectly defined via the connectivity kernel, which itself obeys certain constraints (e.g., limited spatial scale; see below). I believe it is important for the authors to clarify what they mean by "geometry." They should also specify which aspects are under their control, and whether, in fact, all geometries can be realized.
(2) The authors make two key assumptions: that connectivity is purely inhibitory and that the connectivity kernel has a small spatial scale. They state that under these conditions, the homogeneous fixed point becomes unstable, leading to a non-periodic state. However, it seems to me that they do not demonstrate that this emergent state is necessarily a bump localized in all manifold dimensions -- although this is assumed throughout the manuscript. Are other solutions possible or observed? For example, might the network converge to states that are localized in one dimension but extended in another, yielding e.g., stripe-like activity in the plane rather than bumps? In other words, does the proposed recipe guarantee convergence to bumps? This is a critical point and should be clarified.
(3) Related to the question above: What are the failure modes when these two assumptions are violated? Does the network always exhibit runaway activity (as suggested in the text), or can other types of solutions emerge? It would be useful if the authors could briefly discuss this.
(4) Again, related to the question above: can this formalism be extended to activity profiles beyond bumps? For example, periodic fields as seen in grid cells, or irregular fields as observed in many biological datasets -- particularly in naturalistic environments? These activity profiles are of key importance to neuroscientists, so I believe this is an important point that should at least be addressed in the Discussion. Can MADE be naturally extended to these scenarios? What are the challenges involved?
(5) Line 119: "Since σ is the only spatial scale being introduced in the dynamics, we qualitatively expect that a localized bump state within the ball will have a spatial scale of O(σ)." Is this statement always true? I understand that the spatial scale of the synaptic inputs exchanged via recurrent interactions (i.e., the argument of the function f in Equation 1) is characterised by the spatial scale σ. But the non-linear function f could modify that spatial scale -- for example, by "cutting" the bump close to its tip. Where am I wrong? Could the authors clarify?
(6) The authors provide beautiful intuition about the problem of constructing integrators on non-trivial topologies and propose a mathematically grounded solution using Killing vectors. Of course, solutions based on Killing vectors are more complex than those with constant offsets, which raises the question: Is the brain capable of learning and handling such complex structures? Perhaps the authors could speculate in the Discussion about the biological plausibility of these mechanisms.
(7) A great merit of this paper is that it provides mathematical tools for neuroscience researchers to build integrators on non-trivial geometries. I found that, although all the necessary information is present in the Methods, the authors could improve the presentation by schematizing the steps required to build each type of model. It would be extremely useful if, for each considered geometry, the authors provided a short list of required components: the manifold P, the choice of distance, and the connectivity offsets defined by the Killing vectors. Currently, this information is presented, but scattered (not grouped by geometry).
-
Reviewer #2 (Public review):
Summary:
The work by Claudi et al. presents a framework for constructing continuous attractor neural networks (CANs) with user-defined topologies and integration capabilities. The framework unifies and generalizes classical attractor models and includes simulations across a range of topologies, including ring, torus, sphere, Möbius band, and Klein bottle. A key contribution of the paper is the introduction of Killing vectors to enable integration on non-parallelizable manifolds. However, the need for Killing vectors currently appears hypothetical, as biologically discovered manifolds-such as rings and tori-do not require them.
Moreover, throughout the manuscript, the authors claim to be addressing "biologically plausible" attractor networks, yet the constraints required by their construction - such as exact …
Reviewer #2 (Public review):
Summary:
The work by Claudi et al. presents a framework for constructing continuous attractor neural networks (CANs) with user-defined topologies and integration capabilities. The framework unifies and generalizes classical attractor models and includes simulations across a range of topologies, including ring, torus, sphere, Möbius band, and Klein bottle. A key contribution of the paper is the introduction of Killing vectors to enable integration on non-parallelizable manifolds. However, the need for Killing vectors currently appears hypothetical, as biologically discovered manifolds-such as rings and tori-do not require them.
Moreover, throughout the manuscript, the authors claim to be addressing "biologically plausible" attractor networks, yet the constraints required by their construction - such as exact symmetry, fine-tuning of weights, and idealized geometry-seem incompatible with biological variability. It appears that "biologically plausible" is effectively used to mean "capable of integration." While these issues do not diminish the contributions of the work, they should be acknowledged and addressed more explicitly in the text. I applaud the authors for their interesting work. Below are my major and minor concerns.
Strengths:
(1) Theoretical framework for integrating CANs
The paper introduces a systematic method for constructing continuous attractor networks (CANs) with arbitrary topologies. This goes beyond classical models and includes novel topologies such as the Möbius band, sphere, and Klein bottle. The approach generalizes well-known ring and torus attractor models and provides a unified view of their construction, dynamics, and integration capabilities.(2) Novel use of killing vector fields
A key theoretical innovation is the introduction of Killing vectors to support velocity integration on non-parallelizable manifolds. This is mathematically elegant and extends the domain of tractable attractor models.(3) Insightful simulations across manifolds
The paper includes detailed simulations demonstrating bump attractor dynamics across a range of topologies.Weaknesses:
(1) Biological plausibility is overstated
Despite frequent use of the term "biologically plausible," the models rely on assumptions (e.g., symmetric connectivity, perfect geometries, fine-tuning) that are not consistent with known biological networks, and the authors do not incorporate heterogeneity, noise, or constraints like Dale's law.(2) Continuum of states not directly demonstrated
The authors claim to generate a continuum of stable states but do not provide direct evidence (e.g., Jacobian analysis with zero eigenvalues along the manifold). This weakens the central claim about the nature of the attractor.(3) Lack of clarity around assumptions
Several assumptions and analyses (e.g., symmetry breaking, linearity, stability conditions) are introduced without justification or overstated. The analytical rigor in discussing alternative solutions and bifurcation behavior is limited.(4) Scalability to high dimensions
The authors claim their method scales better than learning-based approaches. This should be better discussed.Major Concerns
(1) Biological plausibility
The claim that the proposed framework is "biologically plausible" is misleading, as it is unclear what the authors mean by this term. Biological plausibility could include features such as heterogeneity in synaptic weights, randomness in tuning curves, irregular geometries, or connectivity constraints consistent with known biological architectures (e.g., Dale's law, multiple cell types). None of these elements is implemented in the current framework. Furthermore, it is not clear whether the framework can be extended to include such features-for example, CANs with heterogeneous connections or tuning curves. The connectivity matrix is symmetric to allow an energy-based description and analytical tractability, which is fine, but not a biologically realistic constraint. I recommend removing or significantly qualifying the use of the term "biologically plausible."
(2) Continuum of stable states
While the authors claim their model generates a continuum of stable states, this is not demonstrated directly in their simulations or in a stability analysis (though there are some indirect hints). One way to provide evidence would be to compute the Jacobian at various points along the manifold and show that it possesses (approximately) zero eigenvalues in the tangent/on-manifold directions at each point (e.g., see Ságodi et al. 2024 and others). It would be especially valuable to provide such analysis for the more complex topologies illustrated in the paper.(3) Assumptions, limitations, and analytical rigor
Some assumptions and derivations lack justification or are presented without sufficient detail. Examples include:• Line 126: "If the homogeneous state (all neurons equally active) were unstable, there must exist some other stable state, with broken symmetry." Is this guaranteed? In the ring model with ReLU activation, there could also be unbounded solutions-not just bump solutions-and, in principle, there could also be oscillatory or other solutions. In general, multiple states can co-exist, with differing stability. It appears the authors only analyze the homogeneous case and do not study the stability or bifurcations of other solutions, limiting their theoretical work.
• Line 122: "The conditions for the formation..." What are these conditions, precisely? A citation or elaboration would be helpful. Why is the assumption σ≪L necessary, and how does it impact the construction or conclusions?
• The theory relies heavily on exact symmetries and fine-tuned parameters. Indeed, in line 106, the authors write: "We seek interaction weights consistent with the formation, through symmetry breaking." Is this symmetry-breaking necessary for all CANs? Or is it a limitation specific to hand-crafted models (see also below)? There is insufficient discussion of such limitations. For example, it is difficult to envision how the authors' framework might form attractor manifolds with different geometries or heterogeneous tuning curves.
(4) Comparison with models of learned attractors
While the connectivity patterns of learned attractors often resemble classical hand-crafted models (e.g., see also Vafidis et al. 2022), this is not always the case. If initial conditions include randomness or if the geometry of the attractor deviates from standard forms, the solutions can diverge significantly from hand-designed architectures. Such biologically realistic conditions highlight the limitations the hand-crafted CANs like those proposed here. I suggest updating the discussion accordingly.(5) High-Dimensional Manifolds
The authors argue that their method scales better than training-based approaches in high dimensions and that it is straightforward to extend their framework to generate high-dimensional CANs. It would be useful for the authors to elaborate further. First, it is unclear what k refers to in the expression k^M used in the introduction. Second, trained neural networks seem to exhibit inductive bias (e.g., Cantar et al. 2021; Bordelon & Pehlevan 2022; Darshan & Rivkind 2022), which may mitigate such scaling issues. To support their claim, the authors could also provide an example of a high-dimensional manifold and show that their framework efficiently supports a (semi-)continuum of stable states. -
-
-
