Flexing the principal gradient of the cerebral cortex to suit changing semantic task demands

  1. Zhiyao Gao
  2. Li Zheng
  3. Katya Krieger-Redwood
  4. Ajay Halai
  5. Daniel S Margulies
  6. Jonathan Smallwood
  7. Elizabeth Jefferies

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    This work provides substantial new insights into how semantic association strength influences the function and relationships across brain regions along a topographical structure of cerebral cortex. A principal gradient with the separation of default mode network from sensory-motor systems represents a hallmark of the retrieval of strong conceptual links. This study will be of interest to cognitive neuroscientists, especially those who are interested in semantic cognition.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and Reviewer #3 agreed to share their name with the authors.)

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Understanding how thought emerges from the topographical structure of the cerebral cortex is a primary goal of cognitive neuroscience. Recent work has revealed a principal gradient of intrinsic connectivity capturing the separation of sensory-motor cortex from transmodal regions of the default mode network (DMN); this is thought to facilitate memory-guided cognition. However, studies have not explored how this dimension of connectivity changes when conceptual retrieval is controlled to suit the context. We used gradient decomposition of informational connectivity in a semantic association task to establish how the similarity in connectivity across brain regions changes during familiar and more original patterns of retrieval. Multivoxel activation patterns at opposite ends of the principal gradient were more divergent when participants retrieved stronger associations; therefore, when long-term semantic information is sufficient for ongoing cognition, regions supporting heteromodal memory are functionally separated from sensory-motor experience. In contrast, when less related concepts were linked, this dimension of connectivity was reduced in strength as semantic control regions separated from the DMN to generate more flexible and original responses. We also observed fewer dimensions within the neural response towards the apex of the principal gradient when strong associations were retrieved, reflecting less complex or varied neural coding across trials and participants. In this way, the principal gradient explains how semantic cognition is organised in the human cerebral cortex: the separation of DMN from sensory-motor systems is a hallmark of the retrieval of strong conceptual links that are culturally shared.

Article activity feed

  1. Version published to 10.7554/elife.80368 on eLife
  2. eLife

    Author Response

    Reviewer #2 (Public Review):

    Recent advances in the investigation of functional brain connectivity have allowed the identification of the main connectivity gradient between unimodal to transmodal brain regions. Gao et al. aimed to test whether this connectivity gradient is changing according to task demands and if so, whether this change was also related to the complexity of brain signals evoked by events of various task demands. Their results are three-fold. 1) They first compared the gradient of connectivity obtained during a semantic relatedness judgment task to a purely visual detection task and to a resting state. a) They found that the same main gradient could be extracted from the three conditions, making it suitable for investigating the effect of word relatedness. b) Additionally, they showed that the word relatedness modulates the main gradient: when words are close, the gradient was strengthened, i.e., the dissociation between unimodal and transmodal areas was sharpened. 2) The authors found that the strength of word associations modulates the complexity of brain signals: the closer the words, the more convergent brain signals across participants and trials were, particularly in the transmodal areas of the main gradient. 3) They found that transmodal brain regions in the gradient were similarly activated in participants with similar relatedness judgments. Finally, they tested the link between the three results above using mediation analysis. They showed that the dimensionality difference (result 2) mediated the link between the gradient in the semantic task (result 1a) and the interindividual similarities in semantic judgment and brain activation (result 3). Altogether, this study demonstrates that the main gradient state is predictive of both task variations and inter-individual similarities of task responses. Those results suggest that gradients are a relevant measure of functional connectivity for investigating the variation of connectivity within a task and between individuals. The results overall support conclusions.

    • Strengths:

    1. The main strength of the article is the methods used to obtain the results. Gradients of functional connectivity are a new measure that goes beyond classical brain network functional connectivity. Investigating the dynamics of gradients during a semantic task allows us to better understand how different brain regions (unimodal, transmodal, belonging to some specific networks, etc.) adapt to variability in a task.

    The second strength is the topic: the question is relevant to researchers interested in semantic memory or processing and to any researcher interested in brain dynamics within and between individuals. The demonstration is elegant, and the behavioral task is simple; it compensates for the complexity of the methods.

    • Weaknesses:

    1. The main weakness of the article is the lack of details about the performed analyses, which prevents a clear understanding of the results. The complexity of their methods calls for a crystal-clear description of them. The reader is not informed about how statistics are computed. New terms are sometimes used to describe already mentioned results, making reading the article particularly difficult.

    Thanks very much for the suggestions on statistics. We have now significantly updated our manuscript, please see our detailed reply to Essential Revision.

    1. Conceptually, the authors assumed that during the task, participants generated a word linking the pair of words displayed on the screen and that the neural and cognitive processes solely vary along with the distance between the two words of the pair. However, when words are close, it is not obvious that individuals will generate a third word to link them, and it might be even more challenging to find a linking word in that case as opposed to when words are quite distant from each other. Considering those potential confounds, the interpretation of the results could be different. The authors always contrast very high versus very low distance, then the observed results could also be interpreted as: "observing a link" versus "generating a word link", the first scenario is much more cognitively simple, and this could also explain the differences they observed.

    Sorry that we did not explain our task instruction clearly in our initial submission. The participants were not instructed to generate a linking word specifically and the link was typically expressed in multiple words and could involve imagery as well as words. For this reason, we are not sure that a simple recognition/generation distinction will capture the different neural effects that relate to high and low associations. However, the text now acknowledges that multiple cognitive processes could contribute to the differences we observe, including recognition vs. generation, more automatic retrieval vs. more controlled retrieval, and processes associated with creativity. We have acknowledged multiple ways that the neural patterns could be interpreted in the discussion. Please see page 29.

    ‘Though our results are in line with controlled semantic cognition framework in general, while multiple cognitive processes could contribute to the differences that relate to strong and weak associations we observe, including observing vs. generating semantic links, more automatic retrieval vs. more controlled retrieval, imagery, and processes associated with creativity.’

    Reviewer #3 (Public Review):

    With resting-state fMRI data, recent work has mapped the organisation of the cortex along a continuous gradient, and regions that share similar patterns of functional connectivity are located at similar points on the gradient (Margulies et al., 2016). In the current study, the authors investigate how this dimension of connectivity changes during conceptual retrieval with different levels of semantic association strength. Specifically, they perform gradient analysis on task-fMRI informational connectivity data and reveal a similar principal gradient to the previous study, which captures the separation of heteromodal memory regions from the unimodal cortex. More importantly, by comparing the gradient generated with data from different experimental conditions (i.e., strong vs. weak association), the authors find the separation of the regions at the two ends of the gradient can be regulated by the association strength, with more separation for stronger association. They also examine the relationships between the gradient values and dimensionality and brain-semantic alignment measures, to explore the nature of this shifting gradient as well as the corresponding brain areas.

    Strengths:

    1. The aim of this study is clear and the relevant background literature is covered at an appropriate level of detail. With the cortical gradient analysis approach, this study has the potential to make a contribution to the understanding of the topographical neural basis of semantics in a fine-grained manner.
    1. The methodology in the current study is novel. This study validates the feasibility of performing gradient analysis on task-fMRI data, which is enlightening for future research. Using the number of PCs generated by PCA as a measure of dimensionality is also an interesting approach.
    1. The authors have conducted multiple control analyses, which tested the validity of their results. Specifically, a control task without engaging semantic processing was built in the experimental design (i.e., the chevron task), and the authors conducted multiple parallel control analyses with the data from this control task as a comparison with their main results. Other control analyses were also performed to validate the robustness of their methodological choices. For example, varied thresholds were used during the calculation of dimensionality and similar results were obtained.

    Weaknesses:

    1. As a major manipulation in the experiment, it is not very clear how the authors split/define their stimuli into strong and weak semantic association conditions. If I understood correctly, word2vec was used to measure the association strength in each pair of words. Then the authors grouped the top 1/3 association strength trials as a "strong association" condition and the bottom 1/3 as "weak association" (Line 689), and all analyses comparing the effect of "strong vs. weak association" were conducted with data from these two subsets of stimuli. However, in multiple places, the authors indicate the association strength of their stimuli ranges from completely unrelated to weakly related to highly related (Line 612, Line 147, Line 690, and the examples in Figure 1B). This makes me wonder if the trials with bottom 1/3 association strength (i.e., those were used in the current study) are actually "unrelated/no association" trials (more like a baseline condition), instead of "weak association" trials as the authors claimed. These two situations could be different regarding how they engage semantic knowledge and control processing. Besides, I am very interested in what will the authors find if they compare all three conditions (i.e., unrelated vs. weak association vs. strong association).

    Thanks very much for bringing up this point. We have conducted additional analysis for the intermediary bin and compare it against the bottom for the gradient analysis and against the top 1/3 for the dimensionality analysis (compared to the baseline condition for each analysis), which did show a similar patten like the contrast between strong and weak association but with a smaller effect, thus representing an intermediary profile as expected. The correlation between the principle gradient difference between middle and weak association with the principle gradient value derived from resting state was also significant, see Figure S10C, but its magnitude was smaller than what we reported in the main body of manuscript (r = 0.235 vs. r = 0.369). Given that the expected strongest effect is between top and bottom 1/3, thus, we have now included these results in the supplementary materials. Please see Figure S10 in page 7.

    1. Following the previous point, because the comparison between weak vs. strong association conditions is the key of the current study, I feel it might be better to introduce more about the stimuli in these two conditions. Specifically, the authors only suggested the word pairs fell in these two conditions varied in their association strength, but how about other psycholinguistic properties that could potentially confound their manipulation? For example, words with higher frequency and concreteness may engage more automatic/richer long-term semantic information and words with lower frequency and concreteness need more semantic control. I feel there may be a possibility that the effect of semantic association was partly driven by the differences in these measures in different conditions.

    Thanks for raising this point. We have performed additional control analysis to examine the relationship between association strength and psycholinguistic features according to the reviewer’s suggestion. The association strength did not show significant correlation with word frequency (r = -0.010, p = 0.392), concreteness (r = -0.092, p = 0.285) or imageability (r = 0.074, p = 0.377). Direction comparison of these psycholinguistic features between strongly and weakly associated word-pairs also did not any significant difference: frequency (t = 0.912, p = 0.364), concreteness (t = 1.576, p = 0.119), imageability (t = 1.451, p = 0.153). Please see in page 32:

    ‘The association strength did not show significant correlation with word frequency (r = -0.010, p = 0.392), concreteness (r = -0.092, p = 0.285) or imageability (r = 0.074, p = 0.377).’

    1. The dimensionality analysis in the current study is novel and interesting. In this section, the authors linked decreasing dimensionality with more abstract and less variable representations. However, most results here were built based on the comparison between the dimensionality effects for strong and weak association conditions. I wonder if these conclusions can be generalised to results within each condition and across different regions (i.e., regions having lower dimensionality are doing more abstract and cross-modal processing). If so, I am curious why the ATL (a semantic "hub") in Figure 3A has higher dimensionality than the sensory-motor cortices (quite experiences related) and AG (another semantic "hub").

    The dimensionality and its relationship to the cortical gradient was also examined for each condition. We assessed whether this relationship was influenced by associative strength, averaging dimensionality estimates for sets of four trials with similar word2vec values using a ‘sliding window’ approach. There was a negative correlation between overall dimensionality (averaged across all trials) and principal gradient. And the magnitude of this negative relationship increases as a function of the association strength. So, we believe our conclusion could be generalized across conditions. In our results, we observed higher dimensionality in ATL/frontal orbital cortex than sensory-motor cortices, which seems contradictory to our conclusion. However, these areas are subject to severe distortion and signal loss in functional MRI, the lower tSNR, thus, caused higher dimensionality estimation in PCA. Therefore, we conducted a control analysis in which regions in limbic network were removed due to their low tSNR, while this pattern remained significant (r = -0.346, p = 0.038).

    Please see in Discussion part in page 30.

    ‘It is worth noting that not all brain regions showed the expected pattern in the dimensionality analysis – especially when considering the global dimensionality of all semantic trials, as opposed to the influence of strength of association in the semantic task. In particular, the limbic network, including regions of ventral ATL thought to support a heteromodal semantic hub, showed significantly higher dimensionality than sensory-motor areas – these higher-order regions are expected to show lower dimensionality corresponding to more abstract representations. However, this analysis does not assess the psychological significance of data dimensionality differences (unlike our contrast of strong and weak associations, which are more interpretable in terms of semantic cognition). Limbic regions are subject to severe distortion and signal loss in functional MRI, which might strongly influence this metric. Future studies using data acquisition and analysis techniques that are less susceptible to this problem are required to fully characterize global dimensionality and its relation to the principal gradient.’

    1. I am not sure about the meaning/representational content underlying the semantic similarity matrix in the semantic-brain alignment analysis. According to the authors, this matrix was built based on the correlation of participants' ratings of associative strength (0, no link; 1~4, weak to strong) across trials. The authors indicate that this matrix reflects the global similarity of semantic knowledge between participants (Line 403). However, even though two participants share very similar ratings of association strength across trials, they could still interpret the meaning/knowledge underlying the associations very differently. For example, one participant may interpret the link between "man" and "car" as a man owns a car but another participant may interpret it as a man is hit by a car, although both associations could be rated as strong for this trial. This situation may be even more obvious for those pairs with weak association. Therefore, I am not confident this is a measure of similarity of semantic knowledge.

    Thanks very much for bring up this point. Our experimenter carefully evaluated the links generated for each trial in each participant and found that the weaker association the less consistent their link being formed was. So, we agreed with the reviewer that even when two participants share similar ratings of association strength, they could still interpret those word pairs significantly different, especially for those weakly associated trials. Despite the retrieval content/meaning might be different, i.e. a man owns a car or a man is hit by car, both scenarios are quite consistent and without strong semantic conflict being detected. Therefore, we argued that the semantic-brain alignment might reflect the similarity of neural states of retrieval rather than general semantic content. We have now updated this point in the manuscript. Please see on page 20. ‘A semantic similarity matrix, based on the correlation of participants’ ratings of associative strength across trials (reflecting the global similarity of neural states of retrieval between participants; left-hand panel of Figure 4A), was positively associated with neural pattern similarity in inferior frontal gyrus, posterior middle temporal gyrus, right anterior temporal lobe, bilateral lateral and medial parietal cortex, pre-supplementary motor area, and middle and superior frontal cortex (right-hand panel of Figure 4A).’

  3. eLife

    Evaluation Summary:

    This work provides substantial new insights into how semantic association strength influences the function and relationships across brain regions along a topographical structure of cerebral cortex. A principal gradient with the separation of default mode network from sensory-motor systems represents a hallmark of the retrieval of strong conceptual links. This study will be of interest to cognitive neuroscientists, especially those who are interested in semantic cognition.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 and Reviewer #3 agreed to share their name with the authors.)

  4. eLife

    Reviewer #1 (Public Review):

    This work employs a new method, namely connectivity gradient, for measuring the brain-cognition relationship. Such a method has been proposed and widely studied in large-scale connectivity. It reveals that cortical function and intrinsic connectivity change systematically along a 'principal gradient', which has primary sensory and motor cortex at one end, and transmodal regions implicated in abstract and memory-based functions at the other. Recently it has become possible to detect such gradient associations in humans using task-based fMRI. This paper provides a modelling and inference framework for detecting such gradient-related links to human semantic cognition. Specifically, the authors manipulated the degree to which ongoing semantic cognition was aligned with long-term semantic knowledge and quantified the similarity of the multivariate response to each trial along the principal gradient. Such elegant design should therefore be expected to indicate that the dimensionality of neural representations in a semantic task to decrease from unimodal to transmodal areas along the principal gradient, reflecting increasingly abstract and culturally shared representations towards the apex of the gradient. This work could be a promising flag-use for task-based fMRI brain-cognition association studies using the gradient method.

  5. eLife

    Reviewer #2 (Public Review):

    Recent advances in the investigation of functional brain connectivity have allowed the identification of the main connectivity gradient between unimodal to transmodal brain regions. Gao et al. aimed to test whether this connectivity gradient is changing according to task demands and if so, whether this change was also related to the complexity of brain signals evoked by events of various task demands. Their results are three-fold. 1) They first compared the gradient of connectivity obtained during a semantic relatedness judgment task to a purely visual detection task and to a resting state. a) They found that the same main gradient could be extracted from the three conditions, making it suitable for investigating the effect of word relatedness. b) Additionally, they showed that the word relatedness modulates the main gradient: when words are close, the gradient was strengthened, i.e., the dissociation between unimodal and transmodal areas was sharpened. 2) The authors found that the strength of word associations modulates the complexity of brain signals: the closer the words, the more convergent brain signals across participants and trials were, particularly in the transmodal areas of the main gradient. 3) They found that transmodal brain regions in the gradient were similarly activated in participants with similar relatedness judgments. Finally, they tested the link between the three results above using mediation analysis. They showed that the dimensionality difference (result 2) mediated the link between the gradient in the semantic task (result 1a) and the interindividual similarities in semantic judgment and brain activation (result 3). Altogether, this study demonstrates that the main gradient state is predictive of both task variations and inter-individual similarities of task responses. Those results suggest that gradients are a relevant measure of functional connectivity for investigating the variation of connectivity within a task and between individuals. The results overall support conclusions.

    • Strengths:
    The main strength of the article is the methods used to obtain the results. Gradients of functional connectivity are a new measure that goes beyond classical brain network functional connectivity. Investigating the dynamics of gradients during a semantic task allows us to better understand how different brain regions (unimodal, transmodal, belonging to some specific networks, etc.) adapt to variability in a task.
    The second strength is the topic: the question is relevant to researchers interested in semantic memory or processing and to any researcher interested in brain dynamics within and between individuals. The demonstration is elegant, and the behavioral task is simple; it compensates for the complexity of the methods.

    • Weaknesses:
    The main weakness of the article is the lack of details about the performed analyses, which prevents a clear understanding of the results. The complexity of their methods calls for a crystal-clear description of them. The reader is not informed about how statistics are computed. New terms are sometimes used to describe already mentioned results, making reading the article particularly difficult.
    Conceptually, the authors assumed that during the task, participants generated a word linking the pair of words displayed on the screen and that the neural and cognitive processes solely vary along with the distance between the two words of the pair. However, when words are close, it is not obvious that individuals will generate a third word to link them, and it might be even more challenging to find a linking word in that case as opposed to when words are quite distant from each other. Considering those potential confounds, the interpretation of the results could be different. The authors always contrast very high versus very low distance, then the observed results could also be interpreted as: "observing a link" versus "generating a word link", the first scenario is much more cognitively simple, and this could also explain the differences they observed.

  6. eLife

    Reviewer #3 (Public Review):

    With resting-state fMRI data, recent work has mapped the organisation of the cortex along a continuous gradient, and regions that share similar patterns of functional connectivity are located at similar points on the gradient (Margulies et al., 2016). In the current study, the authors investigate how this dimension of connectivity changes during conceptual retrieval with different levels of semantic association strength. Specifically, they perform gradient analysis on task-fMRI informational connectivity data and reveal a similar principal gradient to the previous study, which captures the separation of heteromodal memory regions from the unimodal cortex. More importantly, by comparing the gradient generated with data from different experimental conditions (i.e., strong vs. weak association), the authors find the separation of the regions at the two ends of the gradient can be regulated by the association strength, with more separation for stronger association. They also examine the relationships between the gradient values and dimensionality and brain-semantic alignment measures, to explore the nature of this shifting gradient as well as the corresponding brain areas.

    Strengths:
    1. The aim of this study is clear and the relevant background literature is covered at an appropriate level of detail. With the cortical gradient analysis approach, this study has the potential to make a contribution to the understanding of the topographical neural basis of semantics in a fine-grained manner.
    2. The methodology in the current study is novel. This study validates the feasibility of performing gradient analysis on task-fMRI data, which is enlightening for future research. Using the number of PCs generated by PCA as a measure of dimensionality is also an interesting approach.
    3. The authors have conducted multiple control analyses, which tested the validity of their results. Specifically, a control task without engaging semantic processing was built in the experimental design (i.e., the chevron task), and the authors conducted multiple parallel control analyses with the data from this control task as a comparison with their main results. Other control analyses were also performed to validate the robustness of their methodological choices. For example, varied thresholds were used during the calculation of dimensionality and similar results were obtained.

    Weaknesses:
    1. As a major manipulation in the experiment, it is not very clear how the authors split/define their stimuli into strong and weak semantic association conditions. If I understood correctly, word2vec was used to measure the association strength in each pair of words. Then the authors grouped the top 1/3 association strength trials as a "strong association" condition and the bottom 1/3 as "weak association" (Line 689), and all analyses comparing the effect of "strong vs. weak association" were conducted with data from these two subsets of stimuli. However, in multiple places, the authors indicate the association strength of their stimuli ranges from completely unrelated to weakly related to highly related (Line 612, Line 147, Line 690, and the examples in Figure 1B). This makes me wonder if the trials with bottom 1/3 association strength (i.e., those were used in the current study) are actually "unrelated/no association" trials (more like a baseline condition), instead of "weak association" trials as the authors claimed. These two situations could be different regarding how they engage semantic knowledge and control processing. Besides, I am very interested in what will the authors find if they compare all three conditions (i.e., unrelated vs. weak association vs. strong association).
    2. Following the previous point, because the comparison between weak vs. strong association conditions is the key of the current study, I feel it might be better to introduce more about the stimuli in these two conditions. Specifically, the authors only suggested the word pairs fell in these two conditions varied in their association strength, but how about other psycholinguistic properties that could potentially confound their manipulation? For example, words with higher frequency and concreteness may engage more automatic/richer long-term semantic information and words with lower frequency and concreteness need more semantic control. I feel there may be a possibility that the effect of semantic association was partly driven by the differences in these measures in different conditions.
    3. The dimensionality analysis in the current study is novel and interesting. In this section, the authors linked decreasing dimensionality with more abstract and less variable representations. However, most results here were built based on the comparison between the dimensionality effects for strong and weak association conditions. I wonder if these conclusions can be generalised to results within each condition and across different regions (i.e., regions having lower dimensionality are doing more abstract and cross-modal processing). If so, I am curious why the ATL (a semantic "hub") in Figure 3A has higher dimensionality than the sensory-motor cortices (quite experiences related) and AG (another semantic "hub").
    4. I am not sure about the meaning/representational content underlying the semantic similarity matrix in the semantic-brain alignment analysis. According to the authors, this matrix was built based on the correlation of participants' ratings of associative strength (0, no link; 1~4, weak to strong) across trials. The authors indicate that this matrix reflects the global similarity of semantic knowledge between participants (Line 403). However, even though two participants share very similar ratings of association strength across trials, they could still interpret the meaning/knowledge underlying the associations very differently. For example, one participant may interpret the link between "man" and "car" as a man owns a car but another participant may interpret it as a man is hit by a car, although both associations could be rated as strong for this trial. This situation may be even more obvious for those pairs with weak association. Therefore, I am not confident this is a measure of similarity of semantic knowledge.

  7. Version published to 10.1101/2022.05.14.491226v2 on bioRxiv
  8. Version published to 10.1101/2022.05.14.491226v1 on bioRxiv