Conditional Dependencies Between Response Time and Item Discrimination: An Item-Level Meta-Analysis

The use of process data such as response time (RT) in psychometrics has generally focused on the relationship between speed and accuracy. The potential relationships between RT and item discrimination remain less explored. In this study, we propose a model for simultaneously estimating the relationships between RT and item discrimination at the person, item, and person-by-item (residual) levels and illustrate our approach through an item-level meta-analysis of 37 empirical datasets comprising approximately 850,000 item responses. We find no evidence of average differences in item discrimination between items of different time intensity or persons of different average RT, while residual RT strongly and negatively predicts item discrimination (pooled coef. = -.31% per 1% difference in RT, SE = .04, tau = .22). While heterogeneity is high and several datasets show strong positive or negative relationships, we find no evidence of moderation by overall dataset characteristics. Flexible generalized additive models show that the relationship between residual RT and item discrimination is generally curvilinear, with discrimination maximized just below average RT and minimized at the extremes. Our results suggest that RT data can provide insights into the measurement properties of educational and psychological assessments, but that the relationships between RT and item discrimination are highly variable.