Weighted Likelihood Estimation of Latent Ability in Sequential Item Response Theory: Properties and Comparisons

Sequential item response theory (SIRT) models have been proposed to deal with item responses in the form of multiple attempts at the same item. They offer several advantages in such settings over traditional polytomous IRT models, such as preserving item characteristic curves across different numbers of attempts and naturally accommodating omitted responses. Despite these benefits, ability estimation for SIRT models has received little attention. It is unclear empirically which ability estimator performs the best under SIRT models with known item parameters and under what conditions. It is also unknown whether the theoretical properties of ability estimators hold under the SIRT models. In this paper, we focus on the Weighted Likelihood (WL) estimator, which has been shown to have nice properties theoretically and empirically under traditional IRT models but is not examined at all in the context of SIRT models. Given that the SIRT models are different from the usual difference-in-cumulative-probability or divide-by-total IRT models, the theoretical and empirical advantages of the WL estimator established previously do not necessarily carry over. In this article, we first develop and formalize the WL estimator for SIRT models. Second, we investigate its theoretical properties, including its relationship to the Jeffreys Modal (JM) estimator and the finiteness of the resulting ability estimates. Third, we compare the performance of WL with Maximum Likelihood (ML), Maximum a Posteriori (MAP), and JM estimators, with particular emphasis on short test lengths. Our theoretical results show that, for sequential 2PL models, the WL estimator is always finite but is not equivalent to the JM estimator. That said, a simulation study indicates that the practical differences between WL and JM are minimal. Across a wide set of conditions, both WL and JM estimators substantially reduce bias relative to ML and often achieve the smallest root mean square error, especially for very short tests and for extreme $\theta$ values.