The Statistical Costs of Two-Step Signal Detection Analyses: A Case for a Maximum Likelihood Mixed-Effects Approach

Signal detection theory (SDT) is a highly influential modeling framework for studying decisions under uncertainty, allowing researchers to disentangle two central components of judgment: Discriminability, the ability to accurately distinguish signal from noise, and response bias, the tendency to favor one type of judgment over the other. The present work compares two approaches for analyzing hierarchically-structured SDT data: The commonly-used two-step approach, which is based on independent individual-level SDT estimates, and a maximum likelihood mixed-effects SDT approach, which uses generalized linear mixed models to estimate individual-level parameters based on a joint population distribution. Across three simulation studies, we demonstrate that the two-step approach produces substantial distortions in both single- and multilevel estimates of discriminability, response criterion and differences therein. These distortions depend in complex ways on the true population means, true population differences, the numbers of trials and participants, and dependencies among individual parameters. Crucially, they also lead to inflated Type I error rates for tests of condition differences in both SDT parameters. In contrast, mixed-effects SDT models yielded effectively unbiased parameter recovery, maintained nominal Type I error rates across a broad range of simulated conditions, and provided higher statistical power for detecting differences in discriminability and response criterion. To facilitate the adoption of this approach,we provide an easy-to-use R package called mesdt and an accompanying tutorial.