Cross-movie prediction of individualized functional topography
Curation statements for this article:-
Curated by eLife
eLife assessment
This valuable study presents a tool for hyperaligning functional brain topography between individuals, which is based on fMRI connectivity data gathered when participants watched different movies. The tool is validated through strong correlations between functional topographic maps generated from a participant's own localizer data and those derived from other participants' data based on this hyperalignment, even when the training and target participants were drawn from different datasets. The study will potentially be of interest to researchers working with a wide range of fMRI datasets.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Participant-specific, functionally defined brain areas are usually mapped with functional localizers and estimated by making contrasts between responses to single categories of input. Naturalistic stimuli engage multiple brain systems in parallel, provide more ecologically plausible estimates of real-world statistics, and are friendly to special populations. The current study shows that cortical functional topographies in individual participants can be estimated with high fidelity from naturalistic stimuli. Importantly, we demonstrate that robust, individualized estimates can be obtained even when participants watched different movies, were scanned with different parameters/scanners, and were sampled from different institutes across the world. Our results create a foundation for future studies that allow researchers to estimate a broad range of functional topographies based on naturalistic movies and a normative database, making it possible to integrate high-level cognitive functions across datasets from laboratories worldwide.
Article activity feed
-
-
Author Response
Reviewer #1 (Public Review):
Due complicated and often unpredictable idiosyncratic differences, comparing fMRI topography between subjects typically would require extra expensive scan time and extra laborious analyzing steps to examine with specific functional localizer scan runs that contrast fMRI responses of every subject to different stimulus categories. To overcome this challenge, hyperaligning tools have recently been developed (e.g., Guntupalli et al., 2016; Haxby et al., 2011) based on aligning in a high-dimensional space of voxels of subjects' fMRI responses to watching a given movie. In the present study, Jiahui and colleagues propose a significantly improved version of hyperaligning functional brain topography between individuals. This new version, based on fMRI connectivity, works robustly on datasets when …
Author Response
Reviewer #1 (Public Review):
Due complicated and often unpredictable idiosyncratic differences, comparing fMRI topography between subjects typically would require extra expensive scan time and extra laborious analyzing steps to examine with specific functional localizer scan runs that contrast fMRI responses of every subject to different stimulus categories. To overcome this challenge, hyperaligning tools have recently been developed (e.g., Guntupalli et al., 2016; Haxby et al., 2011) based on aligning in a high-dimensional space of voxels of subjects' fMRI responses to watching a given movie. In the present study, Jiahui and colleagues propose a significantly improved version of hyperaligning functional brain topography between individuals. This new version, based on fMRI connectivity, works robustly on datasets when subjects watched different movies and were scanned with different parameters/scanners at different MRI centers.
Robustness is the major strength of this study. Despite the fact that datasets from different subjects watching different movies at different MRI centers with different scan parameters were used, the results of functional brain topography from between-subject hyperalignment based on fMRI connectivity were comparable to the golden standard of within-subject functional localizations, and significantly better than regular surface anatomical alignments. These results also support the claim that the present approach is a useful improvement from previous hyperalignments based on time-locked fMRI voxel responses, which would require normative samples of subjects watching a same movie.
We thank the reviewer for the appreciation of our work.
Given the robustness, this new version of hyperalignment would provide much stronger statistical power for group-level comparisons with less costs of time and efforts to collect and analyze data from large sample size according to the current stringent standard, likely being useful to the whole research community of functional neuroimaging. That said, more discussions of the limit of the present hyperalignment approach would be helpful to potential eLife readers. For example, to what extend the present hyperalignment approach would be applicable to individuals with atypical functional brain topography such as brain lesion patients with e.g., acquired prosopagnosia? Even in typical populations, while bilateral fusiform face areas can be identified in the majority through functional localizer scans, the left fusiform face area sometimes cannot be found. Moreover, many top-down factors are known to modulate functional brain topography. Due to these factors, brain responses and functional connectivity may be different even when a same subject watched a same movie twice (e.g., Cui et al., 2021).
We thank the reviewer for the suggestion and agree that it would be fascinating if the predictions can be made with high fidelity in neuropsychological populations. Although we are optimistic that our algorithm is able to generalize across diverse populations, to date, no previous literature has provided empirical evidence to illustrate the effectiveness, including optimizations and special applications beyond typical brains. Besides the neuropsychological population, it would also be valuable to study the generalization across a broad age range, for example, from infants to the elderly. The brain changes across age both anatomically and functionally, so it is a challenge to predict functional topographies based on a normative group that only includes young participants. With all these potential applications in mind, future research is needed to illustrate the efficacy, build the pipeline, and construct the representative normative groups to meet the requirements of accurate individualized predictions in diverse populations.
In typical populations, although participants have great individual variabilities in their functional topographies, for instance, some participants have distinguishable patches of activations in their left ventral temporal cortex while some participants don’t, our algorithms successfully captured these individualized differences in the prediction. The figure below shows, as an example, the face-selective topographies of two individuals that have markedly different face-selective topographies on the left ventral temporal cortex. The left participant has prominent face-selective areas on the left ventral temporal cortex that are in similar sizes as the right side, while the right participant only has a few scattered small face-selective spots on the left side. No matter what their face-selective areas look like, our algorithm accurately recovered the individualized locations, shapes, and sizes, retaining the individual variability in the functional topographies.
Functional connectivity profiles based on naturalistic stimuli are very stable across the cortex, even when participants watch different movies. In Figure 4-figure supplement 9, the mean correlations of fine-scaled connectome for most searchlights (r = 15mm) when participants watched The Grand Budapest Hotel and the Raiders of the Lost Ark were generally around 0.8. The mean correlations were about 0.9 between the first and second half of the same movie although the stimuli contents were different between the two halves. Thus, the fine-grained functional connectivity profiles remain highly stable and reliable across movie contents, which contributes to the robustness of cross-movie, time, and other parameters (e.g., scanner models, scanning parameter) predictions using our algorithms.
We added a paragraph in the discuss section to address the concerns (page 18-19):
“This study successfully illustrated that accurate individualized predictions are both robust and applicable across a variety of conditions, including movie types, languages, scanning parameters, and scanner models. Importantly, the intricate connectivity profiles remain consistent even when participants view entirely different movies, as evidenced by Figure 4-figure supplement 9, reinforcing the prediction's stability in various scenarios. However, all four datasets in this study only included typical participants with anatomically intact brains. An unanswered question is whether individualized topographies of neuropsychological populations with atypical cortical function (e.g., developmental prosopagnosics) or with lesioned brains (e.g., acquired prosopagnosics) could also be accurately predicted using the hyperalignment-based methods. Up to now, as far as we know, no previous literature has investigated this question. Beyond neuropsychological groups, it is also valuable to investigate how well the predictions will be across a wide range of age, from infants to the elderly. Future research is essential to adapt our algorithms to diverse populations.”
Reviewer #2 (Public Review):
Guo and her colleagues develop a new approach to map the category-selective functional topographies in individual participants based on their movie-viewing fMRI data and functional localizer data from a normative sample. The connectivity hyperalignment are used to derived the transformation matrices between the participants according to their functional connectomes during movies watching. The transformation matrices are then used to project the localizer data from the normative sample into the new participant and create the idiosyncratic cortical topography for the participant. The authors demonstrate that a target participant's individualized category-selective topography can be accurately estimated using connectivity hyperalignment, regardless of whether different movies are used to calculate the connectome and regardless of other data collection parameters. The new approach allows researchers to estimate a broad range of functional topographies based on naturalistic movies and a normative database, making it possible to integrate datasets from laboratories worldwide to map functional areas for individuals. The topic is of broad interest for neuroimaging community; the rationale of the study is straightforward and the experiments were well designed; the results are comprehensive. I have some concerns that the authors may want to address, particularly on the details of the pipeline used to map individual category-selective functional topographies.
We thank the reviewer for the encouragement.
- How does the length of the scan-length of movie-viewing fMRI affect the accuracy in predicting the idiosyncratic cortical topography? Similarly, how does the number of participants in the normative database affect the prediction of the category-selective topography? This information is important for the researchers who are interested in using the approach in their studies.
To investigate the influence of movie-viewing data length and the number of participants in the normative database on prediction performance, we systematically varied these parameters. Specifically, we altered the number of runs utilized in the analysis for both the normative and target data and experimented with varying the number of participants in the normative dataset using the Budapest and the Sraiders datasets. We have included a new Figure 4-figure supplement 5 to present a summary of these findings.
The results reveal that both within-dataset and between-dataset prediction performances are positively correlated with the length of movie-viewing fMRI data used for both the normative and target groups. A similar trend was observed with respect to the number of participants included in the normative dataset. It is important to highlight, though, that, even when analyzing as little as one run of movie-viewing data—roughly 10-15 minutes, our hyperalignment-based prediction performance was significantly higher than that achieved using traditional surface alignment. This held true even when the normative dataset included as few as five participants.
In summary, our results show that prediction performance generally improves with longer movie-viewing sessions and larger normative datasets. However, it is noteworthy that even with minimal data—10 minutes of movie-viewing and a small number of participants in the normative dataset—our algorithm still outperforms traditional surface alignment methods significantly.
We also added sentences in the discussion section (page 15):
“We investigated the influence of naturalistic movie length and the size of the training group on the prediction accuracy of individualized functional topographies. By incrementally increasing both the number of movie runs in the training and target dataset and the participants in the training group in the Budapest and Sraiders dataset, we observed enhanced prediction accuracy (Figure 4-figure supplement 5). Notably, even with just one movie run in the training or target dataset, or with a mere five participants in the training group, our prediction performance (Pearson r) ranged from about 0.6 to 0.7. This accuracy significantly outperformed results obtained using surface-based alignment.”
- The data show that category-selective topography can be accurately estimated using connectivity hyperalignment, regardless of whether different movies are used to calculate the connectome and regardless of other data collection parameters. I'm wondering whether the functional connectome from resting state fMRI can do the same job as the movie-watching fMRI. If it is yes, it will expand the approach to broader data.
We agree with the reviewer that demonstrating the applicability of the resting state data will expand the application scenarios of this approach. Most previous findings on resting state connectivity, including the comparison between the naturalistic and the resting state paradigms, focused on the macro-scale similarities and differences (e.g., Samara et al., 2023). Very few studies have investigated the fine-scaled connectome based on resting state data. The study on connectivity hyperalignment (Guntupalli et al., 2018) demonstrated a shared fine-scale connectivity structure among individuals that co-exists with the common coarse-scale structure and built the algorithm to successfully hyperalign individuals to the shared fine-scaled space. Another study from our lab (Feilong et al., 2021) revealed that the fine-scaled connectivity profiles in both resting and task states are highly predictive of general intelligence, indicating reliable and biologically relevant fine-scaled resting state connectome structures. Thus, it is highly plausible that our approach is able to be generalized to the resting state data, generating significantly better predictions of individualized functional topographies than traditional surface alignment. However, due to the limitations of the current datasets, we do not have resting state data available in the current datasets to perform this analysis. We are in the process of collecting new data to explore this hypothesis in future work.
We added sentences to the discussion section to discuss this idea (page 18):
“Studies comparing movie-viewing and resting state functional connectivity have shown that both paradigms yield overlapping macroscale cortical organizations (29), though naturalistic viewing introduces unique modality-specific hierarchical gradients. However, there remains a gap in research comparing the fine-scaled connectomes of naturalistic and resting state paradigms. Guntupalli and colleagues (14) revealed a shared fine-scale structure that coexists with the coarse-scale structure, and connectivity hyperalignment successfully improved intersubject correlations across a wide variety of tasks. Feilong et al. (13) noted that the fine-scaled connectivity profiles in both resting and task states are highly predictive of general intelligence. This suggests a reliable and biologically relevant fine-scale resting state connectivity structure among individuals. Therefore, it is plausible that individualized functional topography could be effectively estimated using resting state functional connectivity, expanding the applicability of our approach. Future studies are needed to explore this direction.”
- The authors averaged the hyper-aligned functional localizer data from all of subjects to predict individual category-selective topographies. As there are large spatial variability in the functional areas across subjects, averaging the data from many subjects may blur boundaries of the functional areas. A better solution might be to average those subjects who show highly similar connectome to the target subjects.
We appreciate the reviewer’s insightful question about optimizing prediction performance by selecting participants most similar in functional connectivity to the target individuals. This is a promising direction and difficult problem as well. Our approach is based on fine-scale connectome to hyperalign participants, thus different groups of participants may be similar to the target participant in different searchlights. In addition, based on results discussed in the response to Q2, the more participants included in the normative dataset, the better the prediction performance. Thus, there is a trade-off between the number of participants included in the normative dataset for the prediction and the overall similarity of those participants to the target participant.
To quantitatively explore this idea, we used a searchlight in the right ventral temporal cortex, roughly at the location of posterior fusiform face area (pFFA).We sorted participants by their connectome similarity to each target participant and then examined prediction performance based on either the top nine most similar participants or the bottom nine least similar participants. Our results, presented in Figure 4-figure supplement 8, reveal that hyperalignment consistently outperforms surface alignment regardless of the subset of participants used. Notably, using the nine most similar participants did not significantly alter prediction performance (Tukey Test, z = -0.09, p = 0.996), while using the least similar participants did negatively impact it (Tukey Test, z = 2.492, p = 0.034). Interestingly, the stability of hyperalignment-based predictions remained high even when only a subset of participants was used, contrasting with the variability observed in surface-alignment-based predictions.
Overall, these findings suggest that while selecting functionally similar participants is a promising avenue for future optimization, the process will require nuanced, searchlight-specific criteria. Each searchlight may necessitate its own set of optimal participants to balance between the performance boost from having more participants and the fidelity gained from participant similarity.
We added the following to the discussion in the manuscript (page 16):
“In our study, we used fine-scale connectomes, noting that some participants are more similar to the target participant in specific searchlights. It is an interesting question whether predictions could be enhanced by exclusively selecting those more similar participants for the target participant. To explore this option, we examined a searchlight in the right ventral temporal cortex that was roughly at the location of the posterior fusiform area (pFFA) using the top and bottom nine participants similar to each target participant measured by their fine-scale connectome similarities in the budapest dataset. Generally, using all or part of the participants for the prediction generated similar results (Figure 4-figure supplement 8). Compared to using all the participants, using only the top nine participants who are the most similar to the target participants did not significantly improve the prediction (Tukey Test, z = -0.09, p = 0.996), but using only the bottom nine participants generated significantly lower prediction accuracies (Tukey Test, z = 2.492, p = 0.034). This suggests a trade-off between the number of participants included in the prediction and the similarity of the participants. Future studies are needed to explore the optimal threshold for the number of participants included for each searchlight to refine the algorithm.”
- It is good to see that predictions made with hyperalignment were close to and sometimes even exceeded the reliability values measured by Cronbach's alpha. But, please clarify how the Cronbach's alpha is calculated.
Cronbach’s alpha calculates the correlation score between localizer-based maps across the runs, and it reflects the amount of noise in maps based on individual localizer runs. Traditionally, the reliability was estimated based on split-half correlations. For example, Guntupalli et al. (2016) used correlations of category-selectivity maps between odd and even localizer runs as the measure of reliability. The odd/even split measure underestimated reliability and necessitated recalculation of correlations between maps for only half the data to provide valid comparisons. In contrast, Cronbach’s alpha involves all localizer runs and provides a more accurate statistical estimate of the reliability of the topographies estimated with localizer runs.
Cronbach’s alpha has been used in many previously published works from our lab (e.g., Feilong et al., 2021; Jiahui et al., 2020, 2023). The code for implementing this metric is publicly accessible on the first author’s Github repository (https://github.com/GUOJiahui/face_DCNN/blob/main/code/cronbach_alpha.py).
We added the detailed explanation above to the Material and Methods section (page 24):
“Cronbach’s alpha calculates the correlation score between localizer-based maps across the runs, and it reflects the amount of noise in maps based on individual localizer runs. Traditionally, the reliability was estimated based on split-half correlations. The common odd/even split measure underestimated reliability and necessitated recalculation of correlations between maps for only half the data to provide valid comparisons. In contrast, Cronbach’s alpha involves all localizer runs and provides a more accurate statistical estimate of the reliability of the topographies estimated with localizer runs.”
- Which algorithm was used to perform surface-based anatomical alignment? Can the state-ofthe-art Multimodal Surface Matching (MSM) algorithm from HCP achieve better performance?
We preprocessed our datasets using fMRIPrep, which employs algorithms from FreeSurfer’s recon-all for surface-based anatomical alignment. It is worth noting that different alignment methods can yield varying degrees of performance. For instance, a study by Coalson et al. (2018) compared the localization performance of multiple surface-based alignment methods, including Multimodal Surface Matching (MSM) and FreeSurfer. The study found that MSM outperformed FreeSurfer in terms of peak probabilities and spatial clustering, suggesting better overall localization.
Additionally, Guntupalli et al. (2018) evaluated intersubject correlations (ISC) of functional connectivity from movie-viewing data using both Connectivity Hyperalignment (CHA) and MSM-All with the Human Connectome Project (HCP) dataset. The study showed that although MSM-All yielded marginally better ISC than traditional surface alignment, CHA’s performance was significantly superior.
In summary, while using a more advanced alignment algorithm like MSM could marginally improve prediction performance, its advantages may not be substantial when compared to our CHA-based predictions. The combination of MSM and CHA represents an intriguing direction for future research, although it falls outside the scope of our current study.
- Is it necessary to project to the time course of the functional localizer from the normative sample into the new participants? Does it work if we just project the contrast maps from the normative samples to the new subjects?
It is an interesting question and a practical alternative to researchers to know whether time series of the localizer runs are required to obtain reasonable predictions, as in some scenarios, contrast maps may be the only accessible data in the analysis. To quantitatively explore this possibility, we applied transformation matrices derived from the movie data to training participants’s individual pre-calculated contrast maps of all four categories, and evaluated the predictions. We found nearly similar prediction performance between the two flavors within and across datasets (Figure 4-figure supplement 7). However, it is worth noting that applying transformation matrices directly to contrast maps did not get as much improvement in the interactive steps as the other flavor in the advanced CHA, perhaps due to the scale changes when multiple iterations were implemented and the difficulty to properly normalize the t-maps compared to the regular time series.
Overall, although our algorithm is originally designed to be used on the time course of the functional localizer runs, relatively comparable results can be generated even when the contrast maps are directly projected from the normative group to the target participant. However, to derive the best results with our approach, time series are recommended when the situation permits.
We have also added the contents into the Discussion section (page 16):
“Our original algorithm is designed to apply transformation matrices to the time series of localizer data of training participants before generating contrast maps. To explore whether directly applying these matrices to pre-calculated contrast maps yields comparable results, we conducted an additional analysis across the four categories. Our findings indicate that the prediction outcomes were indeed quite similar between the two approaches for both the within- and across-datasets predictions (Figure 4-figure supplement 7). However, it is worth noting that the improvements observed with enhanced CHA were not as pronounced when applied directly to the contrast maps as opposed to the time series.”
- Saygin and her colleagues have demonstrated that structural connectivity fingerprints can predict cortical selectivity for multiple visual categories across cortex (Osher DE et al, 2016, Cerebral Cortex; Saygin et al, 2011, Nat. Neurosci). I think there's a connection between those studies and the current study. If the author can discuss the connection between them, it may help us understand why CHA work so well.
We thank the reviewer for raising this point that provides us with the chance of clarifying how our approach differs with methods previously reported in the literature. The computational logic underlying our approach is that we derived the transformation matrices between the training and the target participants in the high-dimensional space based on functional connectivity calculated from the movie data. Then, we applied these transformation matrices to the training participant’s localizer data to accomplish the prediction. On the other hand, Saygin and colleagues directly used diffusion-weighted imaging (DWI) data and predicted participants’ functional responses based on the anatomical-functional correspondence. They evaluated the prediction by calculating the mean absolute errors (AE) of the difference between the actual and predicted contrast responses. Although AE linearly increases with the quality of the prediction, it is difficult to measure the prediction performance of the shape, size, and location of the functional areas precisely using this mean value. With our algorithm, we were able to predict the general location and size of the areas and recover the individualized shapes, generating more powerful predictions. We also used the searchlight analysis to evaluate the performance across the cortex systematically. In addition, Osher et al. (2016) and Saygin et al. (2012) always have a few participants failing to show better predictions based on the connectivity than the group averaged method. Our algorithm is more stable, as all participants across all four datasets had better predicted performance using our algorithm than using the group average. However, although we did not directly use the anatomical-functional correspondence with DWI, the relationships between individual structural connectivity and cortical visual category selectivity could be one of the biological underpinnings that contribute to this robust and accurate prediction.
The Connectivity-Based Shared Response Model (cSRM, Nastase et al., 2020) offers an alternative framework for aligning individuals through functional connectivity. While the overarching aim of cSRM and our methodology converges, substantial differences emerge in the respective implementation and application between the two methods that make our approach the more suitable for predicting individualized topographies. The most significant difference between the two is that, instead of focusing on within-individual connectivity profiles, cSRM used inter-subject functional connectivity (ISFC) in the initial step. This design requires that all participants must have time-locked time series, making the algorithm unusable for cross-content prediction and making it incompatible with resting-state data. Our approach, on the other hand, does not require time-locked stimuli, thereby offering a more flexible framework that permits generalization across different types of stimuli and experimental settings and enables bringing data across laboratories across the world together. Secondly, cSRM predominantly focuses on Region of Interest (ROI) analyses, whereas our model employs searchlight-based analyses designed to comprehensively cover the entire cortical sheet. Whole-brain coverage is needed to generate the topography that reflects the patterns across the cortex. Finally, with the optimized 1step method, our approach directly hyeraligns the training and target participants together, avoiding the accumulation of errors from the intermediate common space. cSRM, with an implementation similar to the classic connectivity hyperalignment, creates and hyperaligns all participants to a shared information space. In summary, while our approach and cSRM share a similar theoretical foundation, our approach has been specifically optimized to address the challenges and complexities in predicting individualized whole-brain functional topographies. Moreover, our approach demonstrates a remarkable ability to generalize across a variety of contexts and stimuli, offering a significant advantage in dealing with diverse experimental settings and datasets.
We have added the contents to the discussion section (page 16-17):
“By leveraging transformation matrices obtained from hyperaligning participants based on movie-viewing data, we successfully mapped these relationships to the training participants’ localizer data, enabling robust predictions. Prior work employing diffusion-weighted imaging (DWI) has underscored the link between anatomical connectivity and category selectivity across diverse visual fields (22, 23) and has established a notable congruence between structural and functional connectivities (24). These findings suggest that the unique anatomical connectivity patterns of individuals may serve as a foundational mechanism, contributing to the stable finescale functional connectome that underpins our approach. The connectivity-based Shared Response Model (cSRM) proposed by Nastase and colleagues (25) used connectivity to functionally align individuals similar to the connectivity hyperalignment algorithm. While both approaches share overarching goals, they diverge considerably in implementation and application. First and most important, cSRM used inter-subject functional connectivity (ISFC) rather than within-subject functional connectivity to initially estimate the connectome. As a result, cSRM requires participants to have time-locked fMRI time series. Therefore, unlike our algorithm, the cSRM approach does not support cross-content applications and also is not suitable for use with resting-state data. Second, cSRM is implemented based on a predefined cortical parcellation rather than the overlapping, regularly-spaced cortical searchlights applied in our method which are not constrained by areal borders. For the application, cSRM has mainly been used to do ROI analysis rather than the estimation of the whole-brain topography that requires broader coverage of the cortex with a searchlight analysis. Third, our method is specifically designed to work in each individual’s space, while cSRM decomposes data across subjects into shared and subjectspecific transformations, focusing on a communal connectivity space. In summary, although cSRM presents a promising alternative for similar aims, its current implementation precludes it from fulfilling the range of applications for which our method is optimized.”
Reviewer #3 (Public Review):
In this paper, Jiahui and colleagues propose a new method for learning individual-specific functional resonance imaging (fMRI) patterns from naturalistic stimuli, extending existing hyperalignment methods. They evaluate this method - enhanced connectivity hyperalignment (CHA) - across four datasets, each comprising between nine (Raiders) and twenty (Budapest, Sraiders) participants.
The work promises to address a significant need in existing functional alignment methods: while hyperalignment and related methods have been increasingly used in the field to compare participants scanned with overlapping stimuli (or lack thereof, in the case of resting state data), their use remains largely tied to naturalistic stimuli. In this case, having non-overlapping stimuli is a significant constraint on application, as many researchers may have access to only partially overlapping stimuli or wish to compare stimuli acquired under different protocols and at different sites.
It is surprising, however, that the authors do not cite a paper that has already successfully demonstrated a functional alignment method that can address exactly this need: a connectivitybased Shared Response Model (cSRM; Nastase et al., 2020, NeuroImage). It would be relevant for the authors to consider the cSRM method in relation to their enhanced CHA method in detail. In particular, both the relative predictive performance as well as associated computational costs would be useful for researchers to understand in considering enhanced CHA for their applications.
We thank the reviewer for raising this point that provides us with the chance of clarifying how our approach differs with methods previously reported in the literature. The computational logic underlying our approach is that we derived the transformation matrices between the training and the target participants in the high-dimensional space based on functional connectivity calculated from the movie data. Then, we applied these transformation matrices to the training participant’s localizer data to accomplish the prediction. On the other hand, Saygin and colleagues directly used diffusion-weighted imaging (DWI) data and predicted participants’ functional responses based on the anatomical-functional correspondence. They evaluated the prediction by calculating the mean absolute errors (AE) of the difference between the actual and predicted contrast responses. Although AE linearly increases with the quality of the prediction, it is difficult to measure the prediction performance of the shape, size, and location of the functional areas precisely using this mean value. With our algorithm, we were able to predict the general location and size of the areas and recover the individualized shapes, generating more powerful predictions. We also used the searchlight analysis to evaluate the performance across the cortex systematically. In addition, Osher et al. (2016) and Saygin et al. (2012) always have a few participants failing to show better predictions based on the connectivity than the group averaged method. Our algorithm is more stable, as all participants across all four datasets had better predicted performance using our algorithm than using the group average. However, although we did not directly use the anatomical-functional correspondence with DWI, the relationships between individual structural connectivity and cortical visual category selectivity could be one of the biological underpinnings that contribute to this robust and accurate prediction.
The Connectivity-Based Shared Response Model (cSRM, Nastase et al., 2020) offers an alternative framework for aligning individuals through functional connectivity. While the overarching aim of cSRM and our methodology converges, substantial differences emerge in the respective implementation and application between the two methods that make our approach the more suitable for predicting individualized topographies. The most significant difference between the two is that, instead of focusing on within-individual connectivity profiles, cSRM used inter-subject functional connectivity (ISFC) in the initial step. This design requires that all participants must have time-locked time series, making the algorithm unusable for cross-content prediction and making it incompatible with resting-state data. Our approach, on the other hand, does not require time-locked stimuli, thereby offering a more flexible framework that permits generalization across different types of stimuli and experimental settings and enables bringing data across laboratories across the world together. Secondly, cSRM predominantly focuses on Region of Interest (ROI) analyses, whereas our model employs searchlight-based analyses designed to comprehensively cover the entire cortical sheet. Whole-brain coverage is needed to generate the topography that reflects the patterns across the cortex. Finally, with the optimized 1step method, our approach directly hyeraligns the training and target participants together, avoiding the accumulation of errors from the intermediate common space. cSRM, with an implementation similar to the classic connectivity hyperalignment, creates and hyperaligns all participants to a shared information space. In summary, while our approach and cSRM share a similar theoretical foundation, our approach has been specifically optimized to address the challenges and complexities in predicting individualized whole-brain functional topographies. Moreover, our approach demonstrates a remarkable ability to generalize across a variety of contexts and stimuli, offering a significant advantage in dealing with diverse experimental settings and datasets.
We have added the contents to the discussion section (page 16-17):
“By leveraging transformation matrices obtained from hyperaligning participants based on movie-viewing data, we successfully mapped these relationships to the training participants’ localizer data, enabling robust predictions. Prior work employing diffusion-weighted imaging (DWI) has underscored the link between anatomical connectivity and category selectivity across diverse visual fields (22, 23) and has established a notable congruence between structural and functional connectivities (24). These findings suggest that the unique anatomical connectivity patterns of individuals may serve as a foundational mechanism, contributing to the stable finescale functional connectome that underpins our approach. The connectivity-based Shared Response Model (cSRM) proposed by Nastase and colleagues (25) used connectivity to functionally align individuals similar to the connectivity hyperalignment algorithm. While both approaches share overarching goals, they diverge considerably in implementation and application. First and most important, cSRM used inter-subject functional connectivity (ISFC) rather than within-subject functional connectivity to initially estimate the connectome. As a result, cSRM requires participants to have time-locked fMRI time series. Therefore, unlike our algorithm, the cSRM approach does not support cross-content applications and also is not suitable for use with resting-state data. Second, cSRM is implemented based on a predefined cortical parcellation rather than the overlapping, regularly-spaced cortical searchlights applied in our method which are not constrained by areal borders. For the application, cSRM has mainly been used to do ROI analysis rather than the estimation of the whole-brain topography that requires broader coverage of the cortex with a searchlight analysis. Third, our method is specifically designed to work in each individual’s space, while cSRM decomposes data across subjects into shared and subjectspecific transformations, focusing on a communal connectivity space. In summary, although cSRM presents a promising alternative for similar aims, its current implementation precludes it from fulfilling the range of applications for which our method is optimized.”
With this in mind, I noted several current weaknesses in the paper:
First, while the enhanced CHA method is a promising update on existing CHA techniques, it is unclear why this particular six step, iterative approach was adopted. That is: why was six steps chosen over any other number? At present, it is not clear if there is an explicit loss function that the authors are minimizing over their iterations. The relative computational cost of six iterations is also likely significant, particularly compared to previous hyperalignment algorithms. A more detailed theoretical understanding of why six iterations are necessary-or if other researchers could adopt a variable number according to the characteristics of their data-would significantly improve the transferability of this method.
In the advanced connectivity hyperalignment implementation, we gradually increased the number of targets. The six steps were not intentionally chosen but were the result of the increase to the maximum number of fine-grained targets, namely single cortical vertices.
Our datasets were resampled to the cortical mesh with 18,742 vertices across both hemispheres (approximately 3 mm vertex spacing; icoorder 5; 20,484 vertices before removing non-cortical vertices). Step 1 was the classic standard connectivity hyperalignment implementation based on the anatomically-aligned data. Since using dense connectivity targets (e.g., using all 18742 vertices on the surface) with anatomically-aligned data generates poor functional correspondence across participants (Busch et al., 2021), we used 1,284 vertices (icoorder 3, before removing the medial wall) as connectivity targets in step 1. However, it is beneficial to include more targets for calculating connectivity patterns after the first iteration of connectivity hyperalignment and repeated iterations to lead to a better solution by gradually aligning the information at finer scales. To better align across participants, we iterated the alignment for another two times (step 2 and step 3) with the same number of 1,284 coarse connectivity targets to ensure improved alignment before increasing the number of targets in the later steps. In step 4, we increased the number of targets to 5,124 (icoorder 4, before removing the medial wall), and iterated with this number of vertices for two times in total (step 4 & step 5) before using all vertices as targets. In the final step (step 6), all vertices were used as connectivity targets.
It is true that the multiple iteration steps largely increased the computational complexity compared to the classic connectivity hyperalignment, but the prediction increase was steady across all datasets and became comparable to response hyperalignment performance which requires time-locked stimuli. We did not use an explicit loss function in the algorithm, but followed the natural progression of the number of potential connectivity targets in the implementation. On the other hand, the difference between the performance of the improved and the classic connectivity hyperalignment was relatively small (difference of r < 0.05), which indicates the effectiveness of our classic algorithm. It is up to the researchers’ own options to adopt the number of iterations and the pace of increasing the number of targets in each step. If computational resources are limited or if a shorter total computational time is the primary priority, using the classic connectivity hyperalignment may be the best option to balance the trade-offs.
The Materials and Methods section had the details of the implementation (page 22-23):
“Using dense connectivity targets (e.g., using all 18742 vertices on the surface) with anatomically-aligned data usually generates poor functional correspondence across participants (33). It is, however, beneficial to include more targets for calculating connectivity patterns after the first iteration of connectivity hyperalignment and repeated iterations to lead to a better solution by gradually aligning the information at finer scales.
We used six steps to further improve the connectivity hyperalignment method. Step 1 was the initial connectivity hyperalignment step as described above that was based on the raw anatomically aligned movie data. The resultant transformation matrices were applied to those movie runs, and the hyperaligned data were then used in step 2 to calculate new connectivity patterns and calculate new transformation matrices. We repeated this procedure iteratively six times and derived transformation matrices for each step. In steps 1, 2, and 3, 642 × 2 (icoorder3, before removing the medial wall) connectivity targets were defined with 13 mm searchlights. In step 4 and 5, 2562 × 2 (icoorder 4, before removing the medial wall) connectivity targets were used with 7 mm searchlights to calculate target mean time series. In the final step 6, all 18742 vertices were included as separate connectivity targets, using each vertex’s time series rather than calculating the mean in a searchlight. Each step of this advanced connectivity hyperalignment algorithm increased the prediction performance (Figure 4-figure supplement 2).”
But to help the readers understand the logic of the advanced connectivity hyperalignment algorithm used in this study, we expanded the discussion section (page 15):
“Because using dense connectivity targets (e.g., using all vertices as connectivity targets) with anatomically-alignment data often leads to suboptimal alignment across participants (33), we started with coarse connectivity targets and gradually increased the number of connectivity targets to form a denser representation of connectivity profiles. The iterations improved the prediction performance step by step, and at the final step (step 6, all vertices were used as connectivity targets) in this analysis, the enhanced CHA generated comparable performance with RHA (Figure 4-figure supplement 4).”
Second, the existing evaluations for enhanced CHA appear to be entirely based on imagederived correlations. That is, the authors compare the predicted image from CHA with the ground-truth image using correlation. While this provides promising initial evidence, correlation-based measures are often difficult to interpret given their sensitivity to image characteristics such as smoothness. Including Cronbach's alpha reliability as a baseline does not address this concern, as it is similarly an image-based statistic. It would be useful to see additional predictive experiments using frameworks such as time-segment classification, intersubject decoding, or encoding models.
We appreciate the reviewer’s concern regarding the stability of local correlations in relation to image characteristics. To address this, we conducted additional analysis using different searchlight sizes (with radii of 10 mm, 15 mm, and 20 mm) to evaluate the predicted categoryselective maps, focusing specifically on the Budapest dataset. The local correlations between the predicted category-selective maps (obtained using enhanced CHA) and participants’ own maps based on classic localizer runs were calculated for each searchlight. We averaged these correlations across participants and plotted the resulting maps, as shown in Figure 4-figure supplement 10. Although using a larger searchlight radius is similar to employing a larger smoothing kernel, the results remained relatively stable across different searchlight sizes, particularly in regions selectively responsive to the specific category. This stability suggests that while the evaluation may be influenced by image-related features, the conclusion would remain consistent under varying parameters.
As for the use of enhanced CHA, it serves as an optimized version of the classic CHA, specifically designed for predicting individualized functional topographies. Evaluating prediction performance in our study is based on t-value contrast maps for each participant. Given this, it's unclear how time-segment classification or other decoding/encoding models could be appropriately implemented for performance evaluation. However, prior research from our lab has already established the effectiveness of classic CHA. Specifically, Guntupalli et al. (2018) showed that classic CHA significantly improved intersubject correlations (ISC) of connectivity profiles across the cortex. They also revealed that CHA captured fine-scale variations in connectivity profiles for nearby cortical nodes across participants and led to improved betweensubject multivariate pattern classification accuracies (bsMVPC) of movie segments. These findings serve as robust evidence for the effectiveness of classic CHA, laying the groundwork for our enhanced CHA approach.
We added Figure 4-figure supplement 10 to the supplementary material:
Addressing these concerns and considering cSRM as a comparison model would significantly strengthen the paper. There are also notable strengths that I would encourage the authors to further pursue. In particular, the authors have access to a unique dataset in which the same Raiders of the Lost Ark stimulus was scanned for participants within the Budapest (SRaiders) dataset as well as non-overlapping participants in the Raiders dataset. Exploring the relative performance for cross-movie prediction within a dataset as compared to a shared movie prediction across datasets is particularly interesting for methods development. I would encourage the authors to explicitly report results in this framework to highlight both this unique testing structure as well as the performance of their enhanced CHA method.
We appreciate the reviewer's suggestion to examine a shared time-series but non-overlapping participants scenario using the Sraiders and Raiders datasets. However, there are significant differences between the two datasets that preclude such direct comparison. These differences include varying scanning parameters, MRI scanners, localizer types, and data collection procedures. Due to these methodological divergences, the datasets cannot be treated as identical time-series.
Firstly, the scanning parameters vary considerably. Sraiders were scanned with TR = 1 s (TR/TE = 1000/33 ms, flip angle = 59 °, resolution = 2.5 mm3 isotropic voxels, matrix size = 96 × 96, FoV = 240 × 240 mm, multiband acceleration factor = 4, and no in-plane acceleration), and Raiders were scanned with TR = 2.5 s (TR = 2.5 s, TE = 35 ms, Flip angle = 90°, 80 × 80 matrix, FOV = 240 mm × 240 mm, resolution = 0.938 mm × 0.938 mm × 1.0 mm).
Secondly, participants in the Sraiders were scanned with a 3 T S Magnetom Prisma MRI scanner with a 32 channel head coil and the Raiders dataset, collected more than 10 years ago, used a 3T Philips Intera Achieva scanner with an eight-channel head coil.
Thirdly, the stimuli presentations were different. In the Sraiders dataset, the movie Raiders of the Lost Ark was split into eight parts (~15 min each), and the first four parts were watched outside of the scanner prior to the scanning (~56 min). The later four parts were watched in the scanner (57 min) with audio. And in the Raiders dataset, the audio-visual movie was split into eight parts (~15 min each). Participants watched all eight parts in the scanner with audio (one part / per run).
Fourthly and critically, the two datasets included two types of localizers. The Sraiders dataset included dynamic localizer runs, and the Raiders dataset only contained a static localizer that was similarly designed as in the Forrest dataset.
With all four points, it is not suitable to treat the two datasets as identical time-series. The difference in the localizer type is a further issue. The topographies generated from the two types of localizers are dissimilar in many ways. For all categories, the dynamic localizer elicited stronger and broader category-selective activations than the static localizer, and the searchlight analysis showed that the dynamic localizer had higher reliabilities across the cortex, especially in regions that were selectively responsive to the target category. Due to these differences, crossdataset predictions yielded lower correlations than within-dataset predictions. This is not indicative of methodological failure but reflects diverging topographies activated by different localizers.
In the manuscript, we have extensively analyzed cross-dataset predictions (Figure 2-figure supplement 1-Figure 4-figure supplement 4 & 6).
● Figure 2-figure supplement 1 demonstrates that, despite the limitations of cross-localizertype evaluation, both R-to-S (Raiders to Sraiders) and S-to-R (Sraiders to Raiders) predictions significantly outperformed surface alignment methods across categories.
● Figure Figure 2-figure supplement 2 confirms that the prediction performance remained stable across individual participants, underscoring the robustness of our methodology.
● Figure 3-figure supplement 1 & Figure 3-figure supplement 2 display contrast maps generated from both native and alternate localizers, revealing that the maps share similar topographies irrespective of the dataset origin.
● Figure 4-figure supplement 1 presents a correlation analysis of local similarities in R-to-S and S-to-R predictions, highlighting particularly strong correlations in the ventral face regions.
● Figure 4-figure supplement 2 employs histograms to showcase performance across major cortices and furnishes additional evidence regarding the influence of localizer types on the results.
● Figure 4-figure supplement 3 offers a searchlight analysis for other categories, enriching the scope of our investigation.
● Figure 4-figure supplement 4 affirms that the advanced CHA is effective in both R-to-S and S-to-R predictions.
● Figure 4-figure supplement 6 compares the efficacy of 1-step vs. 2-step prediction methods for R-to-S and S-to-R, showing a clear advantage for the 1-step approach.
These analyses affirmed that our approach outperforms surface alignment methods. But the inherent limitations in data collection and localizer types preclude a direct exploration of the reviewer’s hypothesis. These complexities necessitate further research to fully validate the proposed scenario.
Overall, I share the authors' enthusiasm for the potential of cross-movie, cross-dataset prediction, and I believe that methods such as enhanced CHA are likely to significantly improve our ability to make these comparisons in the near future. At present, however, I find that the theoretical and experimental support for enhanced CHA is incomplete. It is therefore difficult to assess how enhanced CHA meets its goals or how successfully other researchers would be able to adopt this method in their own experiments.
We hope our new analysis and replies addressed the reviewer’s concerns.
-
eLife assessment
This valuable study presents a tool for hyperaligning functional brain topography between individuals, which is based on fMRI connectivity data gathered when participants watched different movies. The tool is validated through strong correlations between functional topographic maps generated from a participant's own localizer data and those derived from other participants' data based on this hyperalignment, even when the training and target participants were drawn from different datasets. The study will potentially be of interest to researchers working with a wide range of fMRI datasets.
-
Reviewer #1 (Public Review):
Due complicated and often unpredictable idiosyncratic differences, comparing fMRI topography between subjects typically would require extra expensive scan time and extra laborious analyzing steps to examine with specific functional localizer scan runs that contrast fMRI responses of every subject to different stimulus categories. To overcome this challenge, hyperaligning tools have recently been developed (e.g., Guntupalli et al., 2016; Haxby et al., 2011) based on aligning in a high-dimensional space of voxels of subjects' fMRI responses to watching a given movie. In the present study, Jiahui and colleagues propose a significantly improved version of hyperaligning functional brain topography between individuals. This new version, based on fMRI connectivity, works robustly on datasets when subjects watched …
Reviewer #1 (Public Review):
Due complicated and often unpredictable idiosyncratic differences, comparing fMRI topography between subjects typically would require extra expensive scan time and extra laborious analyzing steps to examine with specific functional localizer scan runs that contrast fMRI responses of every subject to different stimulus categories. To overcome this challenge, hyperaligning tools have recently been developed (e.g., Guntupalli et al., 2016; Haxby et al., 2011) based on aligning in a high-dimensional space of voxels of subjects' fMRI responses to watching a given movie. In the present study, Jiahui and colleagues propose a significantly improved version of hyperaligning functional brain topography between individuals. This new version, based on fMRI connectivity, works robustly on datasets when subjects watched different movies and were scanned with different parameters/scanners at different MRI centers.
Robustness is the major strength of this study. Despite the fact that datasets from different subjects watching different movies at different MRI centers with different scan parameters were used, the results of functional brain topography from between-subject hyperalignment based on fMRI connectivity were comparable to the golden standard of within-subject functional localizations, and significantly better than regular surface anatomical alignments. These results also support the claim that the present approach is a useful improvement from previous hyperalignments based on time-locked fMRI voxel responses, which would require normative samples of subjects watching a same movie.
Given the robustness, this new version of hyperalignment would provide much stronger statistical power for group-level comparisons with less costs of time and efforts to collect and analyze data from large sample size according to the current stringent standard, likely being useful to the whole research community of functional neuroimaging. That said, more discussions of the limit of the present hyperalignment approach would be helpful to potential readers. For example, to what extend the present hyperalignment approach would be applicable to individuals with atypical functional brain topography such as brain lesion patients with e.g., acquired prosopagnosia? Even in typical populations, while bilateral fusiform face areas can be identified in the majority through functional localizer scans, the left fusiform face area sometimes cannot be found. Moreover, many top-down factors are known to modulate functional brain topography. Due to these factors, brain responses and functional connectivity may be different even when a same subject watched a same movie twice (e.g., Cui et al., 2021).
-
Reviewer #2 (Public Review):
Guo and her colleagues develop a new approach to map the category-selective functional topographies in individual participants based on their movie-viewing fMRI data and functional localizer data from a normative sample. The connectivity hyperalignment are used to derived the transformation matrices between the participants according to their functional connectomes during movies watching. The transformation matrices are then used to project the localizer data from the normative sample into the new participant and create the idiosyncratic cortical topography for the participant. The authors demonstrate that a target participant's individualized category-selective topography can be accurately estimated using connectivity hyperalignment, regardless of whether different movies are used to calculate the …
Reviewer #2 (Public Review):
Guo and her colleagues develop a new approach to map the category-selective functional topographies in individual participants based on their movie-viewing fMRI data and functional localizer data from a normative sample. The connectivity hyperalignment are used to derived the transformation matrices between the participants according to their functional connectomes during movies watching. The transformation matrices are then used to project the localizer data from the normative sample into the new participant and create the idiosyncratic cortical topography for the participant. The authors demonstrate that a target participant's individualized category-selective topography can be accurately estimated using connectivity hyperalignment, regardless of whether different movies are used to calculate the connectome and regardless of other data collection parameters. The new approach allows researchers to estimate a broad range of functional topographies based on naturalistic movies and a normative database, making it possible to integrate datasets from laboratories worldwide to map functional areas for individuals. The topic is of broad interest for neuroimaging community; the rationale of the study is straightforward and the experiments were well designed; the results are comprehensive. I have some concerns that the authors may want to address, particularly on the details of the pipeline used to map individual category-selective functional topographies.
1. How does the length of the scan-length of movie-viewing fMRI affect the accuracy in predicting the idiosyncratic cortical topography? Similarly, how does the number of participants in the normative database affect the prediction of the category-selective topography? This information is important for the researchers who are interested in using the approach in their studies.
2. The data show that category-selective topography can be accurately estimated using connectivity hyperalignment, regardless of whether different movies are used to calculate the connectome and regardless of other data collection parameters. I'm wondering whether the functional connectome from resting state fMRI can do the same job as the movie-watching fMRI. If it is yes, it will expand the approach to broader data.
3. The authors averaged the hyper-aligned functional localizer data from all of subjects to predict individual category-selective topographies. As there are large spatial variability in the functional areas across subjects, averaging the data from many subjects may blur boundaries of the functional areas. A better solution might be to average those subjects who show highly similar connectome to the target subjects.
4. It is good to see that predictions made with hyperalignment were close to and sometimes even exceeded the reliability values measured by Cronbach's alpha. But, please clarify how the Cronbach's alpha is calculated.
5. Which algorithm was used to perform surface-based anatomical alignment? Can the state-of-the-art Multimodal Surface Matching (MSM) algorithm from HCP achieve better performance?
6. Is it necessary to project to the time course of the functional localizer from the normative sample into the new participants? Does it work if we just project the contrast maps from the normative samples to the new subjects?
7. Saygin and her colleagues have demonstrated that structural connectivity fingerprints can predict cortical selectivity for multiple visual categories across cortex (Osher DE et al, 2016, Cerebral Cortex; Saygin et al, 2011, Nat. Neurosci). I think there's a connection between those studies and the current study. If the author can discuss the connection between them, it may help us understand why CHA work so well. -
Reviewer #3 (Public Review):
In this paper, Jiahui and colleagues propose a new method for learning individual-specific functional resonance imaging (fMRI) patterns from naturalistic stimuli, extending existing hyperalignment methods. They evaluate this method - enhanced connectivity hyperalignment (CHA) - across four datasets, each comprising between nine (Raiders) and twenty (Budapest, Sraiders) participants.
The work promises to address a significant need in existing functional alignment methods: while hyperalignment and related methods have been increasingly used in the field to compare participants scanned with overlapping stimuli (or lack thereof, in the case of resting state data), their use remains largely tied to naturalistic stimuli. In this case, having non-overlapping stimuli is a significant constraint on application, as …
Reviewer #3 (Public Review):
In this paper, Jiahui and colleagues propose a new method for learning individual-specific functional resonance imaging (fMRI) patterns from naturalistic stimuli, extending existing hyperalignment methods. They evaluate this method - enhanced connectivity hyperalignment (CHA) - across four datasets, each comprising between nine (Raiders) and twenty (Budapest, Sraiders) participants.
The work promises to address a significant need in existing functional alignment methods: while hyperalignment and related methods have been increasingly used in the field to compare participants scanned with overlapping stimuli (or lack thereof, in the case of resting state data), their use remains largely tied to naturalistic stimuli. In this case, having non-overlapping stimuli is a significant constraint on application, as many researchers may have access to only partially overlapping stimuli or wish to compare stimuli acquired under different protocols and at different sites.
It is surprising, however, that the authors do not cite a paper that has already successfully demonstrated a functional alignment method that can address exactly this need: a connectivity-based Shared Response Model (cSRM; Nastase et al., 2020, NeuroImage). It would be relevant for the authors to consider the cSRM method in relation to their enhanced CHA method in detail. In particular, both the relative predictive performance as well as associated computational costs would be useful for researchers to understand in considering enhanced CHA for their applications.
With this in mind, I noted several current weaknesses in the paper:
First, while the enhanced CHA method is a promising update on existing CHA techniques, it is unclear why this particular six step, iterative approach was adopted. That is: why was six steps chosen over any other number? At present, it is not clear if there is an explicit loss function that the authors are minimizing over their iterations. The relative computational cost of six iterations is also likely significant, particularly compared to previous hyperalignment algorithms. A more detailed theoretical understanding of why six iterations are necessary-or if other researchers could adopt a variable number according to the characteristics of their data-would significantly improve the transferability of this method.
Second, the existing evaluations for enhanced CHA appear to be entirely based on image-derived correlations. That is, the authors compare the predicted image from CHA with the ground-truth image using correlation. While this provides promising initial evidence, correlation-based measures are often difficult to interpret given their sensitivity to image characteristics such as smoothness. Including Cronbach's alpha reliability as a baseline does not address this concern, as it is similarly an image-based statistic. It would be useful to see additional predictive experiments using frameworks such as time-segment classification, inter-subject decoding, or encoding models.
Addressing these concerns and considering cSRM as a comparison model would significantly strengthen the paper. There are also notable strengths that I would encourage the authors to further pursue. In particular, the authors have access to a unique dataset in which the same Raiders of the Lost Ark stimulus was scanned for participants within the Budapest (SRaiders) dataset as well as non-overlapping participants in the Raiders dataset. Exploring the relative performance for cross-movie prediction within a dataset as compared to a shared movie prediction across datasets is particularly interesting for methods development. I would encourage the authors to explicitly report results in this framework to highlight both this unique testing structure as well as the performance of their enhanced CHA method.
Overall, I share the authors' enthusiasm for the potential of cross-movie, cross-dataset prediction, and I believe that methods such as enhanced CHA are likely to significantly improve our ability to make these comparisons in the near future. At present, however, I find that the theoretical and experimental support for enhanced CHA is incomplete. It is therefore difficult to assess how enhanced CHA meets its goals or how successfully other researchers would be able to adopt this method in their own experiments.
-
-