From Maps to Models: A Survey on the Reliability of Small Studies of Task-Based fMRI

Task-based functional magnetic resonance imaging is a powerful tool for studying brain function, but neuroimaging research produces ongoing concerns regarding small-sample studies and how to interpret them. Although it is well understood that larger samples are preferable, many situations require researchers to make judgments from small studies, including reviewing the existing literature, analyzing pilot data, or assessing subsamples. Quantitative guidance on how to make these judgments remains scarce. To address this, we leverage the Human Connectome Project's Young Adult dataset to survey various analyses--from regional activation maps to predictive models. We find that, for some classic analyses such as detecting regional activation or cluster peak location, studies with as few as 40 subjects are adequate, although this depends crucially on effect sizes. For predictive modeling, similar sizes can be adequate for detecting whether features are predictable, but at least an order of magnitude more (at least hundreds) may be required for developing consistent predictions. These results offer valuable insights for designing and interpreting fMRI studies, emphasizing the importance of considering effect size, sample size, and analysis approach when assessing the reliability of findings. We hope that this survey serves as a reference for identifying which kinds of research questions can be reliably answered with small-scale studies.