Horizontally Tiled Network of Cortico-Basal Ganglia Modules Performs Reinforcement Learning

The neocortex and basal ganglia nuclei are connected along regions that share the same topography and are arranged side-by-side. Inspired by the anatomical characteristics of the cerebrum, we developed a network in which the modules of the neocortex-basal ganglia unit were arranged in a horizontally tiled manner. By applying this network to reinforcement learning tasks, we demonstrated that reinforcement learning can be achieved through horizontal signals passing between modules. Each module not only performs its calculation but also provides signals to adjacent modules. This lateral transmission takes advantage of the differences in the projection ranges of the three basal ganglia pathways, the direct, indirect, and hyperdirect pathways, which have been examined in physiologic studies. We found that these differences enabled temporal-difference-error computations. This study proposes a novel strategy for information processing based on neocortical-basal ganglia circuits, highlighting the computational significance of their anatomically and physiologically clarified features.