Sample Size Planning in Item Response Theory: A Tutorial

Although item response theory (IRT) models have established psychometric advantages over traditional scoring methods, they remain underutilized in practice. We aim to reevaluate common criticisms of IRT in light of substantive, methodological, and computational advances that have transformed the way psychologists measure, collect, and analyze research data. We summarize the major advantages of IRT over classical test theory and factor analysis and highlight some of the more recent developments. The main part of this tutorial provides a practical introduction to simulation-based sample size estimation in IRT, which is integral for obtaining accurate estimates of item/person parameters, effects, and model fit. A priori sample size estimation is a crucial aspect of study planning, particularly valuable for pre-registration and registered reports. To this end, we present a comprehensive guide with 10 key decisions for setting up Monte Carlo simulations and illustrate the procedure with examples from the fields of educational, personality, and clinical psychology. An extensively annotated and easily customizable syntax is available in an online repository.