eLife assessment
This study presents important new insights into how best to address common problems encountered in the statistical analysis of neural data, including those related to temporal autocorrelations and unknown variables. The authors show that certain approaches, including those using cross-validation and permutation tests, are better than others at controlling error rates, particularly false negatives. At present, the evidence presented is incomplete, including a lack of more rigorous theoretical justifications for the differences observed between the different approaches tested and a focus on p-values without considering effect sizes, but could be improved with substantial revisions that address these issues.