Permutation analysis prior to variable selection greatly enhances robustness of OPLS analysis in small cohorts

The R-workflow roplspvs ( R o rthogonal p rojections of l atent s tructures with p ermutation over v ariable s election) facilitates variable selection, model optimization and significance testing using permutations of OPLS-DA models, with the scaled loadings (p[corr]) as the main metric of significance cutoff. Permutations including (over) the variable selection procedure, prior to (sans), as well as post variable selection are performed. The resulting p- values for the correlation of the model (R ² ) and the cross-validated correlation of the model (Q ² ) sans-, post- and over- variable selection are provided as additional model statistics. These model statistics are useful for determining the true significance level of OPLS models, which otherwise have proven difficult to assess particularly for small sample sizes. Furthermore, we propose a means for estimating the background “noise” level based on permutated false positive rates of R ² and Q ² . This novel metric is then utilized to calculate an adjusted Q ² value. Using a publicly available metabolomics dataset, the advantage of performing permutations over variable selection was demonstrated for small sample sizes. Iteratively reducing the sample sizes resulted in overinflated models with increasing R ² and Q ² , and permutations post variable selection indicated falsely significant models. In contrast, the adjusted Q ² was marginally affected by sample size, and represents a robust estimate of model predictability, and permutations over variable selection showed true significance of the models.