Decoding the brain state-dependent relationship between pupil dynamics and resting state fMRI signal fluctuation
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
Pupil diameter is used as an index of the brain's arousal system, and has traditionally thought to be a non-invasive index of specific neuromodulatory activity. It is therefore been heavily used as a measure in neuroscience. More recent data suggests a more complex picture whereby a pupil dilation might track cocktail of different neuromodulators. This paper provides firm data supporting this view, and introduces the new view that the make-up of this cocktail changes significantly over time. Pupil dynamics are linked with different neuromodulatory centers over different intervals of time. This is clearly important data across a broad range of human and animal systems neuroscience.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Pupil dynamics serve as a physiological indicator of cognitive processes and arousal states of the brain across a diverse range of behavioral experiments. Pupil diameter changes reflect brain state fluctuations driven by neuromodulatory systems. Resting-state fMRI (rs-fMRI) has been used to identify global patterns of neuronal correlation with pupil diameter changes; however, the linkage between distinct brain state-dependent activation patterns of neuromodulatory nuclei with pupil dynamics remains to be explored. Here, we identified four clusters of trials with unique activity patterns related to pupil diameter changes in anesthetized rat brains. Going beyond the typical rs-fMRI correlation analysis with pupil dynamics, we decomposed spatiotemporal patterns of rs-fMRI with principal component analysis (PCA) and characterized the cluster-specific pupil–fMRI relationships by optimizing the PCA component weighting via decoding methods. This work shows that pupil dynamics are tightly coupled with different neuromodulatory centers in different trials, presenting a novel PCA-based decoding method to study the brain state-dependent pupil–fMRI relationship.
Article activity feed
-
-
Author Response:
Reviewer #1 (Public Review):
[...] I do have a couple of concerns.
Major issues:
The BOLD hemodynamic response function is slower than the pupil impulse response function. It seems that the authors did not correct for the "lag" between the two (as in Yellin et al., 2015, for example). How much does this matter for the results?
We thank the reviewer for highlighting the insufficient treatment of the potential “lag” between the two signals. In the initial submission we only compared linear regression prediction scores obtained after introducing shifts in the <-5; 5> s range between the signals and verified that prediction scores were the highest at 0 s lag (Figure 3–figure supplement 1B).
In line with the reviewer’s suggestion, we followed the approach of Yellin et al. (2015) and convolved the pupil diameter size …
Author Response:
Reviewer #1 (Public Review):
[...] I do have a couple of concerns.
Major issues:
The BOLD hemodynamic response function is slower than the pupil impulse response function. It seems that the authors did not correct for the "lag" between the two (as in Yellin et al., 2015, for example). How much does this matter for the results?
We thank the reviewer for highlighting the insufficient treatment of the potential “lag” between the two signals. In the initial submission we only compared linear regression prediction scores obtained after introducing shifts in the <-5; 5> s range between the signals and verified that prediction scores were the highest at 0 s lag (Figure 3–figure supplement 1B).
In line with the reviewer’s suggestion, we followed the approach of Yellin et al. (2015) and convolved the pupil diameter size signals with a hemodynamic response function (HRF) and repeated both the prediction and clustering analyses. As the pupil response times differ across conditions showing e.g. a 1 s response to luminance changes in Yellin et al. (2015) and a ~3 s task-evoked response in de Gee et al. (2017), we (similarly as in our previous publication - Pais et al. (2020)) convolved pupil signals with a range of HRFs with different peak times (HRFs shown in Figure 2–figure supplement 3A).
We show that when using the convolved signals the results are reproducible based on the overlap of trials’ cluster membership (Figure 2–figure supplement 3B) as well as high similarities of the cluster-based spatial correlation maps (Figure 2–figure supplement 3C).
In parallel to the signal shift-based prediction results included in the initial submission, we predicted the convolved pupil signals based on PCA fMRI time courses and verified that the highest prediction scores were obtained when predicting signals convolved with a kernel with a peak time at 0 s (and Figure 2–figure supplement 3C). We hypothesize that the slight increase in prediction scores (compared to raw data prediction) is a result of the convolution-based temporal smoothing.
Baseline pupil size was different between the identified clusters. How was pupil size normalized across rats and scanning runs, so that we can meaningfully interpret such a difference?
We thank the reviewer for pointing out the missing information. As part of the pupil diameter extraction pipeline, the diameter was normalized based on the eye size. In each video, the eye size was calculated based on manual landmark identification. Both the eye size and diameter were measured in the number of pixels. The eye size was then used to normalize the pupil size, such that pupil diameter values were limited to the <0, 1> range with 1 being the eye size. We now included this information in the Methods - Pupillometry acquisition & pupil diameter extraction section (Page 12). The PSDs were based on these signals, meaning that cluster baseline differences reflect differences in the mean pupil diameter. As already mentioned in Methods, the signals were variance normalized for the prediction procedure.
A substantial part of the literature focuses on the relationship between task-evoked pupil and neuromodulatory responses. I understand that this paper describes results from a resting state experiment, but even in these conditions one typically observes rapid dilations. Right now, it seems that the analysis is somewhat blind to these. See for example Fig. 2C in which frequencies are plotted only until 0.05Hz. Can we see this on log-log axes, to inspect the higher frequencies? Note that there is some work that indicates that the slower pupil fluctuations more reliably track ACh signaling, and faster fluctuations more reliably track NE signaling (Reimer & McGinley et al., 2016).
We thank the reviewer for pointing out the potentially meaningful analysis direction. In our work, we do not observe different correlation features when comparing e.g. the 1-10Hz rapid pupil dynamics versus the slow oscillation, which could be potentially caused by the anesthetic effect (Discussion, Page 10). Also, it should be noted that the acquired pupillometry data were limited by the quality of recorded videos and by the pupil size extraction method. The low SNR of videos recorded simultaneously with fMRI measurements using an MR-compatible camera made the faster pupil size changes hard to track. Additionally, the fast pupil size changes observed in the extracted signals are to a large degree caused by the fact that DeepLabCut, the employed toolbox, independently marks the landmarks in each video frame. Due to the lack of temporal dependence in landmark identification combined with the low SNR, “by default” landmark locations slightly differed in neighboring frames. This difference was amplified with increasing pupil size. Consequently, the magnitude of pupil size changes faster than 1 Hz was highly correlated with pupil size (i.e. baseline fluctuation). The effect can be seen in raw pupil data plots in Figure 2–figure supplement 2. Consequently, when we extracted e.g. the fluctuation of the 1-10 Hz pupil size power changes and used it to generate the PCA linear regression maps, the maps closely resembled the baseline-based maps (see the all trial-based maps in Response Figure 2B; spatial correlation r=0.83). Simultaneously, the prediction scores of the band-based signals were lower than those based on raw downsampled signals.
The authors write "Cluster 2 had the strongest positive weights in […], but also in brainstem arousal-regulating locus coeruleus, laterodorsal tegmental and parabrachial nuclei." However, the voxel size is very large with respect to the size subcortical nuclei. Because of this, here and in other places, I think the authors should use locus coeruleus region or area, to indicate that their voxel captures more tissue than just LC proper. A discussion paragraph on the spatial specificity of their effects would also help.
Now we explicitly write about the “area/region containing the locus coeruleus” at every mention of LC being highlighted in the maps.
The approach is very data driven and the Results section mostly descriptive. I'm personally not at all unsympathetic to this approach, but I do think the authors could aid the reader better by briefly interpreting their results already in the Results section. Related, the authors end each paragraph with "These results verified […]" or "These results highlight […]"; however they don't explicitly inform us how.
We modified the mentioned sentences to be more descriptive and self-explanatory.
Rainbow and jet colormaps are confusing because they are not perceptually uniform (https://colorcet.holoviz.org/). Please consider using something like "coolwarm"?
We changed all jet colormaps to coolwarm.
Minor issues:
"Trial" is not well defined. I take this is a 15 minute run?
We thank the reviewer for pointing out the missing information. Now, at the first mention of “trial” in the Introduction, we specify the 15-minute trial duration.
How many trials in each cluster (Fig. 2)?
The clusters had the following trial counts: n1=8; n2=30; n3=24; n4=12. We now included this information in the Results section.
It would be nice to see a more zoomed in version of Fig. 5 so that we can actually see the subcortical regions in more detail.
We now provide linear regression PCA maps for each cluster in a bigger size and with overlaid atlas region borders as individual figure supplements (same format as the Figure 4 map created using all trials; Figure 5–figure supplements 1-4).
Reviewer #2 (Public Review):
[...] The mechanisms behind the time-varying fMRI-pupil coupling exhibited under anesthesia could also be further clarified. Specifically:
The clusters appear to involve interpretable brain regions. However, a more formal analysis of reproducibility of these clusters, and statistical testing against an appropriate null model, are not present. Such tests would be useful for establishing the validity of the derived clusters, ensuring that the conclusions are strongly supported. Similarly, the differentiation between power spectral density of each cluster is not yet supported by statistical testing.
We now addressed the essential issue of cluster reproducibility in a series of analysis steps. In the initial submission, we selected the n=4 cluster result based on silhouette scores computed after single initializations of UMAP dimensionality reduction and GMM clustering. Now, we repeated the random initializations 100 times. The selection of n=4 clusters based on silhouette scores has been reproduced (Figure 2B). Instead of selecting cluster memberships from a single initialization, we identified the most common cluster membership for each trial across repetitions. This resulted in changing cluster memberships for 2 out of 74 trials compared to the initial submission. The ratio of label matching and correlation map similarity across the 100 repetitions are shown in Figure 2–figure supplement 1AB).
Next, following the reviewer’s recommendations we performed a spilt-half analysis and compared our results against a null model similarly to Allen et al. (2014). We divided the trials in two random halves 100 times and repeated the clustering analysis. We showed that to a large degree the cluster memberships are preserved when using trial halves (Figure 2–figure supplement 4). Next, using spatial surrogate maps with spatial autocorrelations, and value distributions matching those of real correlation maps (Figure 2–figure supplement 5A; created using the Brainsmash toolbox – Burt (2020)), we verified that the spatial location of correlation values and not the mean values or spatial autocorrelation properties were driving the clustering (Figure 2–figure supplement 5BC).
We also assessed cluster reproducibility using pupil signals convolved with HRF kernels with different peak times (Figure 2–figure supplement 3A) to accommodate for the possible lag between the pupil and fMRI signals. In Figure 2–figure supplement 3BC we show that the clusters were reproducible when using the convolved signals.
With regard to the decoding models, it appears there could be interdependence between the training and testing data (the PCA step seems to include all scans, and it was not clear if the training/testing sets contained data drawn from the same animal).
We thank the reviewer for pointing out the missing information, which we now included in the manuscript (Page 4).
The PCA model was fit only on the 64 training trials. The fit model was then used to project the time courses of the 10 remaining trials onto the existing components. The 64 training trials were randomly chosen and could belong to any rat. We now specify this in the manuscript. Additionally, we repeated the prediction procedure (including the PCA step) on 100 more random train-test data splits. The scatter plot of mean train and validation scores shows that our initial selection is not an extreme value and is representative of the distribution (Figure 3 – figure supplement 1A).
While the paper is motivated by discussion that pupil diameter changes are complex and related to rich behaviors (mental effort, decision making, etc.), this paper examines data from anesthetized rats. The mechanisms behind the time-varying changes in fMRI-pupil coupling in the current data, and the potential impact of anesthesia, were not clear and could be elaborated upon.
We elaborated upon the use and potential impact of anesthetics in a separate paragraph on page 9.
-
Reviewer #2 (Public Review):
In this work, Sobczak et al. suggest that correlations between fMRI and pupil diameter vary over time, and propose an approach to identify distinct clusters of such correlation patterns. The proposed methods are applied to data acquired from anesthetized rats. Based on the clusters obtained, the authors conclude that pupil dynamics are linked with different neuromodulatory centers over different intervals of time.
Overall, I believe that the study is novel and uncovers potential new modes of coupling between neuromodulatory nuclei and pupil diameter. However, additional analysis may be needed to fully support the validity of the derived clusters, and the decoding methods may need some modification before the accuracy values can be properly interpreted. The mechanisms behind the time-varying fMRI-pupil …
Reviewer #2 (Public Review):
In this work, Sobczak et al. suggest that correlations between fMRI and pupil diameter vary over time, and propose an approach to identify distinct clusters of such correlation patterns. The proposed methods are applied to data acquired from anesthetized rats. Based on the clusters obtained, the authors conclude that pupil dynamics are linked with different neuromodulatory centers over different intervals of time.
Overall, I believe that the study is novel and uncovers potential new modes of coupling between neuromodulatory nuclei and pupil diameter. However, additional analysis may be needed to fully support the validity of the derived clusters, and the decoding methods may need some modification before the accuracy values can be properly interpreted. The mechanisms behind the time-varying fMRI-pupil coupling exhibited under anesthesia could also be further clarified. Specifically:
The clusters appear to involve interpretable brain regions. However, a more formal analysis of reproducibility of these clusters, and statistical testing against an appropriate null model, are not present. Such tests would be useful for establishing the validity of the derived clusters, ensuring that the conclusions are strongly supported. Similarly, the differentiation between power spectral density of each cluster is not yet supported by statistical testing.
With regard to the decoding models, it appears there could be interdependence between the training and testing data (the PCA step seems to include all scans, and it was not clear if the training/testing sets contained data drawn from the same animal).
While the paper is motivated by discussion that pupil diameter changes are complex and related to rich behaviors (mental effort, decision making, etc.), this paper examines data from anesthetized rats. The mechanisms behind the time-varying changes in fMRI-pupil coupling in the current data, and the potential impact of anesthesia, were not clear and could be elaborated upon.
-
Reviewer #1 (Public Review):
Human and animal work over the last couple of years established that fluctuations in pupil size track the activity of a number of neuromodulatory nuclei, including the noradrenergic locus coeruleus, cholinergic basal forebrain, serotonergic dorsal raphe and perhaps the dopaminergic midbrain. In other words, pupil size fluctuations might track a "cocktail" of neuromodulators. The current paper leverages sophisticated data driven analysis techniques to show that pupil size changes can indeed be modulated by different combinations of subcortical nuclei. Doing so, the paper helps laying a solid and nuanced neurophysiological foundation for the interpretation of results from cognitive pupillometry, an area of neuroscience and psychology that is rapidly expanding over the past years. I do have a couple of concerns.
Reviewer #1 (Public Review):
Human and animal work over the last couple of years established that fluctuations in pupil size track the activity of a number of neuromodulatory nuclei, including the noradrenergic locus coeruleus, cholinergic basal forebrain, serotonergic dorsal raphe and perhaps the dopaminergic midbrain. In other words, pupil size fluctuations might track a "cocktail" of neuromodulators. The current paper leverages sophisticated data driven analysis techniques to show that pupil size changes can indeed be modulated by different combinations of subcortical nuclei. Doing so, the paper helps laying a solid and nuanced neurophysiological foundation for the interpretation of results from cognitive pupillometry, an area of neuroscience and psychology that is rapidly expanding over the past years. I do have a couple of concerns.
Major issues:
The BOLD hemodynamic response function is slower than the pupil impulse response function. It seems that the authors did not correct for the "lag" between the two (as in Yellin et al., 2015, for example). How much does this matter for the results?
Baseline pupil size was different between the identified clusters. How was pupil size normalized across rats and scanning runs, so that we can meaningfully interpret such a difference?
A substantial part of the literature focuses on the relationship between task-evoked pupil and neuromodulatory responses. I understand that this paper describes results from a resting state experiment, but even in these conditions one typically observes rapid dilations. Right now, it seems that the analysis is somewhat blind to these. See for example Fig. 2C in which frequencies are plotted only until 0.05Hz. Can we see this on log-log axes, to inspect the higher frequencies? Note that there is some work that indicates that the slower pupil fluctuations more reliably track ACh signaling, and faster fluctuations more reliably track NE signaling (Reimer & McGinley et al., 2016).
The authors write "Cluster 2 had the strongest positive weights in [...], but also in brainstem arousal-regulating locus coeruleus, laterodorsal tegmental and parabrachial nuclei." However, the voxel size is very large with respect to the size subcortical nuclei. Because of this, here and in other places, I think the authors should use locus coeruleus region or area, to indicate that their voxel captures more tissue than just LC proper. A discussion paragraph on the spatial specificity of their effects would also help.
The approach is very data driven and the Results section mostly descriptive. I'm personally not at all unsympathetic to this approach, but I do think the authors could aid the reader better by briefly interpreting their results already in the Results section. Related, the authors end each paragraph with "These results verified [...]" or "These results highlight [...]"; however they don't explicitly inform us how.
Rainbow and jet colormaps are confusing because they are not perceptually uniform (https://colorcet.holoviz.org/). Please consider using something like "coolwarm"?
Minor issues:
"Trial" is not well defined. I take this is a 15 minute run?
How many trials in each cluster (Fig. 2)? It would be nice to see a more zoomed in version of Fig. 5 so that we can actually see the subcortical regions in more detail.
-
Evaluation Summary:
Pupil diameter is used as an index of the brain's arousal system, and has traditionally thought to be a non-invasive index of specific neuromodulatory activity. It is therefore been heavily used as a measure in neuroscience. More recent data suggests a more complex picture whereby a pupil dilation might track cocktail of different neuromodulators. This paper provides firm data supporting this view, and introduces the new view that the make-up of this cocktail changes significantly over time. Pupil dynamics are linked with different neuromodulatory centers over different intervals of time. This is clearly important data across a broad range of human and animal systems neuroscience.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private …
Evaluation Summary:
Pupil diameter is used as an index of the brain's arousal system, and has traditionally thought to be a non-invasive index of specific neuromodulatory activity. It is therefore been heavily used as a measure in neuroscience. More recent data suggests a more complex picture whereby a pupil dilation might track cocktail of different neuromodulators. This paper provides firm data supporting this view, and introduces the new view that the make-up of this cocktail changes significantly over time. Pupil dynamics are linked with different neuromodulatory centers over different intervals of time. This is clearly important data across a broad range of human and animal systems neuroscience.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)
-
