Do complex psychometric analyses really matter? Comparing multiple approaches using individual participant data from antidepressant trials

Psychometric methods can be used to reduce redundancy and error in existing measures, albeit different approaches can produce different results. This study aimed to determine the implications of applying different psychometric methods for clinical trial outcomes. Individual participant data from 15 antidepressant treatment trials from Vivli.org were analysed. Baseline (pre-treatment) and 8-week (range 4-12 weeks) outcome data from the Montgomery-Asberg Depression Rating Scale (MADRS) were subjected to best-practice factor analysis (FA), item response theory (IRT) and network analysis (NA) approaches. Trial outcomes for the original summative scores and psychometric-model scores were assessed using multilevel models. Percentage differences in Cohen’s d effect sizes for the original summative and psychometrically-modelled scores were the effects of interest. Each method produced unidimensional models but the modified scales varied from 7-10 items. Treatment effects were unchanged for IRT (10 items), decreased by 1.3%-2.8% for NA (8 items), and increased by 11%-12.5% for FA (7 items). IRT and NA yielded negligible differences in effect outcomes relative to original trials. FA increased effect sizes and may be the most effective method for identifying the items on which placebo and treatment group outcomes differ.