Replay as structural inference in the hippocampal-entorhinal system

  1. Talfan Evans
  2. Neil Burgess

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Model-based decision making relies on the construction of an accurate representation of the underlying state-space, and localization of one’s current state within it. One way to localize is to recognize the state with which incoming sensory observations have been previously associated. Another is to update a previous state estimate given a known transition. In practice, both strategies are subject to uncertainty and must be balanced with respect to their relative confidences; robust learning requires aligning the predictions of both models over historic observations. Here, we propose a dual-systems account of the hippocampal-entorhinal system, where sensory prediction errors between these models during online exploration of state space initiate offline probabilistic inference. Offline inference computes a metric embedding on grid cells of an associative place graph encoded in the recurrent connections between place cells, achieved by message passing between cells representing non-local states. We provide testable explanations for coordinated place and grid cell ‘replay’ as efficient message passing, and for distortions, partial rescaling and direction-dependent offsets in grid patterns as the confidence weighted balancing of model priors, and distortions to grid patterns as reflecting inhomogeneous sensory inputs across states.

Author Summary

  • Minimising prediction errors between transition and sensory input (observation) models predicts partial rescaling and direction-dependent offsets in grid cell firing patterns.

  • Inhomogeneous sensory inputs predict distortions of grid firing patterns during online localisation, and local changes of grid scale during offline inference.

  • Principled information propagation during offline inference predicts coordinated place and grid cell ‘replay’, where sequences propagate between structurally related features.

Article activity feed

  1. eLife

    Reviewer #3:

    This work provides a computational model to explain the change of grid cell firing field structure due to changes in environmental features. It starts from a framework in which self-motion information and those related to external sensory cues are integrated for position estimation. To implement this theoretical modeling framework, it examines grid cell firing as a position estimate, which is derived from place cell firing representing sensory inputs and noisy, self-motion inputs. Then, it adapts this model to explain experimental findings in which the environment partially changed. For example, the rescaling of an environment leads to a disruption of this estimation because the sensory cue and self-motion information misalign. Accordingly, the model describes mechanisms through which the grid cell position estimate is updated when self-motion and hippocampal sensory inputs misalign in this situation. The work also suggests that coordinated replay between hippocampal place cells and entorhinal grid cells provide means to realign the sensory and self-motion cues for accurate position prediction. Probably the strongest achievement of this work is that it developed a biology-based Bayesian inference approach to optimally use both sensory and self-motion information for accurate position estimation. Accordingly, these findings could be useful in related machine learning fields.

    Major comment:

    The work seems to provide a significant advance in computational neuroscience with possible implications to machine learning using brain-derived principles. The major weakness, however, is that it is not written in a way that the majority of neuroscientists (who do not work in this immediate computational field) could benefit from. It often does not explain why/how it came to some conclusions or what those conclusions actually mean - for example, right in the introduction, "This process can also be viewed as an embedding of sensory experience within a low-dimensional manifold (in this case, 2D space), as observed of place cells during sleep". It also does not provide a sufficiently detailed qualitative explanation of the mathematical formulations or what the model actually does at a given condition. So my recommendation would be to carefully rewrite the work to make it readable for a wider audience. I also fear that the work also assumes significant a priori neuroscience information, so people in machine learning fields would not benefit from this work in its current form either.

    It is not clear why place cell input was chosen as sensory input. Place cells also alter their firing with geometry, sensory and contextual changes. Although grid cells require place cell input, place cell firing represents more than just sensory inputs. In fact, they may be more sensitive to non-sensory behavioral, contextual changes than grid cells. Moreover, like grid cells, they are sensitive to self-motion inputs, e.g., speed-sensitivity and, at least in virtual environments, head-direction sensitivity. This point would deserve a detailed discussion.

  2. eLife

    Reviewer #2:

    This paper uses a clever application of the well known Simultaneous Localization and Mapping model (+ replay) to the neuroscience of navigation. The authors capture aspects of the relationship between EC-HPC that are often not captured within one paper/model. Here online prediction error between the EC/HPC systems in the model trigger offline probabilistic inference, or the fast propagation of traveling waves enabling neural message passing between place and grid cell representing non-local states. The authors thus model how such replay - i.e. fast propagation of offline traveling waves passing messages between EC/HP - leads to inference and explains the function of coordinated EC-HP replay. I enjoyed reading the paper and the supplementary material.

    First, I'd like to say that I am impressed by this paper. Second, I see my job as a reviewer merely to give suggestions to help improve the accessibility and clarity of the present manuscript. This could help the reader appreciate a beautiful application of SLAM to HPC-EC interactions as well as the novelty of the present approach in bringing in a number of HPC-EC properties together in one model.

    1. The introduction is rather brief and lacks citations standard for this field. This is understandable as it may be due to earlier versions having been prepared for NeurIPS. It may be helpful if the authors added a bit more background to the introduction so readers can orient themselves and localize this paper in the larger map of the field. It would be especially helpful to repeat this process not only in the intro but throughout the text even if the authors have already cited papers elsewhere, since the authors are elegantly bringing together various different neuroscientific concepts and findings, such as replay, structures, offline traveling waves, propagation speed, shifter cell, etc. A bigger picture intro will help the reader be prepared for all the relevant pieces that are later gradually unfolded.

    It would be especially helpful to offer an overall summary of the main aspects of HPC-EC literature in relation to navigation that will later appear. This will frontload the larger, and in my opinion clever narrative, of the paper where replay, memory, and probabilistic models meet to capture aspects of the literature not previously addressed.

    1. The SLAM (simultaneous localization and mapping) model is used broadly in mobile phones, robotics, automotive, and drones. The authors do not introduce SLAM to the reader, and SLAM (in broad strokes) may not be familiar to potential readers. Even for neuroscientists who may be familiar with SLAM, it may not be clear from the paper which aspects of it are directly similar to existing other models and which aspects are novel in terms of capturing HPC/EC findings. I would strongly encourage an entire section dedicated to SLAM, perhaps even a simple figure or diagram of the broader algorithm. It would be especially helpful if the authors could clarify how their structure replay approach extends existing offline SLAM approaches. This would make the novel approaches in the present paper shine for both bio & ML audiences.

    Providing this big picture will make it easier for the reader to connect aspects of SLAM that are known, with the clever account of traveling waves and other HPC-EC interactions, which are largely overlooked in contemporary models of HPC-EC models of space and structures. It is perhaps also worth to mention RatSLAM, which is another bio-inspired version of SLAM, and the place cell/hippocampus inspiration for SLAM.

    D Ball, S Heath, J Wiles, G Wyeth, P Corke, M Milford, "OpenRatSLAM: an open source brain-based SLAM system", in Autonomous Robots, 34 (3), 149-176, 2013

    1. At first glance, it may appear that there are many moving parts in the paper. To the average neuroscience reader, this may be puzzling, or require going back and forth with some working memory overload to put the pieces together. My suggestion is to have a table of biological/neural functions and the equivalent components of the present model. This guide will allow the reader to see the big picture - and the value of the authors' hard work - in one glance, and be able to look more closely at each section more closely and with the bigger picture in mind. I believe this will only increase the clarity and accessibility of the manuscript.

    2. The authors could perhaps spend a little more time comparing previous modeling attempts at capturing the HP-EC phenomena and traveling through various models, noting caveats of previous models, and advantages and caveats of their model. This could be in the discussion, or earlier, but would help localize the reader in this space a bit better.

    3. Perhaps the authors could briefly clarify where merely Euclidean vs. non-euclidean representations would be expected of the model, and whether they can accommodate >2D maps, e.g. in bats or in nonspatial interactions of HPC-EC.

    4. The discussion could also be improved by synthesizing the old and the new, the significant contribution of this paper and modifications to SLAM, as well as a big picture summary of the various phenomena that come together in the HPC-EC interactions, e.g. via traveling waves.

  3. eLife

    Reviewer #1:

    In the present manuscript, Evans and Burgess present a computational model of the entorhinal-hippocampal network that enables self-localization by learning the correspondence between stimulus position in the environment and internal metric system generated by path integration. Their model is composed of two separate modules, observation and transition, which inform about the relationship between environmental features and the internal metric system, and update the internal metric system between two consecutive positions, respectively. The observation module would correspond to projection from hippocampal place cells (PCs) to entorhinal grid cells (GCs), while the transition module would just update the GCs based on animal's movement. The authors suggest that the system can achieve fast and reliable learning by combining online learning (during exploration) and offline learning (when the animal stops or rests). While online learning only updates the observation model, offline learning could update both modules. The authors then test their model on several environmental manipulations. Finally, they discuss how offline learning could correspond to spontaneous replay in the entorhinal-hippocampal network. While the work will certainly be of great interest to the community, the authors should improve the presentation of their manuscript, and make sure they clearly define the key concepts of their study.

    Online learning is clearly explained in the manuscript (e.g. l.101). Both environment structure (PC-PC connections) and the observation models (PC->GC synapses) are learned online, and this leads to stable grid cells. Then, the authors suggest that prediction error between the observation and transition models triggers offline inference that can update both models simultaneously. However, it is hard to figure out what offline learning is exactly. The section "Offline inference: The hippocampus as a probabilistic graph" is quite impossible to follow. Before explicitly defining offline learning the authors introduce a spring model of mutual connection between feature locations, but it is not clearly explained if this network is optimized online or offline.

    The end of this section is particularly difficult to follow (line 180): "In this context, learning the PC-GC weights (modifying the observation model) during online localization corresponds to forming spatial priors over feature locations which anchor the structure, which would otherwise be translation or rotation invariant (since measurements are relative), learned during offline inference to constant locations on the grid-map.".

    What really triggers offline inference is only explained much further in the manuscript, l. 366. Interestingly, this section refers to Fig. 1G for the first time, and should naturally be moved at the beginning of the manuscript (where Fig.1 is described)

    Along the same lines, the role of offline learning should be made much more explicit in Fig. 2.

    The frequent references to the method section too often break the flow of paper and make it difficult to follow. The authors should start their manuscript with a clear and simple definition of the core idea and concepts, almost in lay terms and only introducing a few annotations, using Fig. 1 (perhaps with some modification and focusing especially on panels A and F) as a visual support, and to move mathematical equations such as Eq. 3 to the supplementary information.

    The authors have tested their model on various manipulations that have been previously carried out in freely moving animals, such as change in visual gain and in environmental geometry. These sections are interesting but, again, would be much clearer if presented after a clear explanation of online and offline learning procedures, not in between.

    Finally, the authors discuss the relationship between offline inference and neuronal replay, as observed experimentally in vivo (Figs 6&7). This is interesting but would perhaps benefit from some graphical explanation. It is not immediately obvious to understand the fundamental difference between message passing (Fig. 6A) and simple synaptic propagation of activity among connected PC in CA3. Fig. 7 is actually a nice illustration of the phenomenon and should perhaps be presented before Fig. 6.

  4. eLife

    Summary: In the present manuscript, the authors apply the well-known Simultaneous Localization and Mapping model (+ replay) to the neuroscience of navigation. Their model is composed of two separate modules, observation and transition. The former informs about the relationship between environmental features and the internal metric system while the latter updates the internal metric system between two consecutive positions. The observation module would correspond to projection from hippocampal place cells to entorhinal grid cells, while the transition module would just update the grid cells based on animal's movement. The authors suggest that the system can achieve fast and reliable learning by combining online learning (during exploration) and offline learning (when the animal stops or rests). In the model, online prediction error between the entorhinal cortex and the hippocampus triggers offline probabilistic inference, during which replay of place and grid cells represent non-local states. The authors thus suggest a function to the experimental observation of coordinated replay in the entorhinal-hippocampal network.

  5. Version published to 10.1101/2020.08.07.241547 on bioRxiv