False discovery rate correction promotes confounded neuroimaging designs
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
False discovery rate corrections are a popular approach to dealing with the problem of multiple comparisons in neuroimaging research. This paper describes a previously undiscussed limitation of false discovery rate corrections. Specifically, the use of false discovery rate will tend to promote the design and publication of poorly controlled experiments. This occurs because the presence of significant hypothesis tests in some parts of the brain will lower the threshold for significance elsewhere in the brain. The inclusion of a major confound in a design will therefore make it easier to find the expected results by artificially lowering the significance threshold relative to a well-controlled study. Here I demonstrate this basic phenomenon using Monte Carlo simulations. I conduct further simulations to estimate how this phenomenon could affect more realistic confounding scenarios and interact with publication bias. I conclude by suggesting mitigation approaches.