Maximizing Memory Capacity in Heterogeneous Networks

A central question in neuroscience is which neuronal and connectivity properties determine a network's ability to store information. The theory of Hopfield models has provided powerful tools to investigate this question. Here, we generalize classical results on the memory capacity of homogeneous Hopfield models to a broader class of heterogeneous networks. We derive an analytical formula for the maximal capacity of networks with arbitrary, and generally heterogeneous, activation rates (coding levels) of neurons in memory patterns, as well as arbitrary connectivity architectures. We use this result to make normative predictions about the properties that maximize capacity in brain-inspired networks. We show that, although heterogeneity in neuron coding levels and inward connection counts (in-degrees) generally reduces capacity, maximal capacity is retained when these two parameters are correlated. This prediction holds across various biologically relevant scenarios, including the storage of independent patterns in both classical and dendritic networks, as well as the storage of example patterns clustered around prototypes representing concepts. In the latter case, heterogeneity similarly affects the capacity for both examples and concepts. Finally, we analyze bipartite models of the CA3-DG circuit in the hippocampus, known to be crucial for memory encoding. In these networks, capacity is maximized by a quasi-indexing encoding scheme, where each neuron in the DG binds subsets of features from a few memory patterns stored in CA3. Compared to a complete-indexing scheme, where each DG unit binds all the features of a single pattern, quasi-indexing significantly improves both network capacity and memory robustness to neuron ablation. These findings introduce new dimensions to the hippocampal index theory of memory encoding and retrieval, while suggesting specific neural substrates for these functions.