Cross-species transcriptome analysis of the subthalamic and para-subthalamic nuclei reveal mRNA patterns unique to primate and relevant to neuropsychiatry

The ventromedial subthalamic nucleus (STN) is associated with affective function via limbic brain networks. Clinical data indicate the anterior aspect of ventromedial STN as an important target in Deep brain stimulation (DBS) intervention of psychiatric disorder, including Obsessive-compulsive disorder (OCD), for which Selective serotonin reuptake inhibitors (SSRIs) are prescribed. However, far from all patients respond to either SSRIs or DBS. Despite clinical relevance, the primate STN is lacking in molecular and anatomical detail. Here, transcriptome data generated in mice was implemented in comparative analysis with macaque and human. Remarkable species differences were observed. Serotonin receptor 2c (HTR2C) mRNA was identified in a distinct ventromedial STN profile in macaque and human, non-paralleled in mouse. This included the medial tip and anteromedial STN, where HTR2C was distributed in patterns matching reports of serotonergic innervation. In primates, but not mice, a previously undescribed far-anterior STN brain area was uncovered, and the anterior, but not posterior, STN was found to border the hypothalamic para-STN. In both mice and primates, gradients of HTR2C mRNA allowed para-STN to be subdivided into core and shell domains. Dorsal to STN, para-STN was partly embedded in the Medial forebrain bundle (MFB), a DBS target in OCD and depression. Transcriptome data visualized in serial sections and 3D brain fly-through videos challenge current atlases of STN, pSTN, and MFB anatomy in mice and primates. Differential mRNA patterns in the primate, including unique HTR2C profiles, may provide openings for drug discovery and enhanced anatomical precision, and uncover neurobiological underpinnings of psychiatric disorders.