NeuroConText: Contrastive Learning for Neuroscience Meta-Analysis with Rich Text Representation
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Coordinate-based meta-analysis (CBMA) is the common way to gather information about human brain function across the existing literature in order to formulate hypotheses and contextualize new findings. However, automated CBMA tools face challenges such as inconsistent terminology and difficulties in analyzing long texts and capturing semantic meaning because they still rely on bag-of-words approaches; furthermore, sparse coordinate reporting in CBMA distorts the activation distribution due to incomplete data. This paper introduces NeuroConText, an automated CBMA tool that bridges neuroscience text, brain location coordinates and brain images by creating a shared latent space for encoding text and brain maps. This relies on a multi-objective loss combining contrastive and reconstruction terms. It leverages large language models (LLMs) to capture neuroscientific information from full-text articles, plus an LLM-based text augmentation strategy to handle short-text inputs. Quantitative and qualitative analyses demonstrate NeuroConText’s ability to enhance text-to-brain retrieval performance and reconstruct brain maps from neuroscience texts. We also show that CBMA tools can infer brain activations in regions discussed in articles but absent in reported coordinates, potentially addressing the challenge of sparse coordinate reporting.
Author summary
Neuroscience studies often suffer from limited sample sizes, resulting in findings with low statistical power and uncertain reproducibility. Coordinate-based meta-analysis (CBMA) addresses this problem by pooling results across studies to identify consistent associations between cognitive processes and brain regions. This article introduces NeuroConText, an automated CBMA tool designed to integrate heterogeneous data sources, including brain images, full-text articles, and reported coordinate tables. NeuroConText uses large language models (LLMs) to capture rich semantic information from full-text of neuroscience articles to link neuroscience text to brain images and coordinates. The goal of NeuroConText is to retrieve brain locations or brain maps associated with text-based descriptions and to reconstruct brain activation patterns from text. NeuroConText is also designed to support short input queries, which makes it applicable to a wider range of use cases. In addition, we show that meta-analysis tools can highlight regions not explicitly reported in coordinate tables, thus overcoming the limitations posed by sparsely reported coordinates in neuroscience articles. In summary, by improving associations between neuroscience texts and brain locations, NeuroConText enhances the capabilities of coordinate-based meta-analysis.
