Deep Learning Cognitive Diagnosis Models for Modeling Response and Process Data under Exam Settings

Deep-learning-based cognitive diagnosis models (CDMs) have been predominantly evaluated on large-scale online tutoring platforms where neural architectures benefit from high-dimensional features and abundant data, raising questions about whether their reported advantages persist in traditional psychometric assessment settings. This paper addresses two questions: how do deep-learning and psychometric CDMs compare in predictive accuracy and attribute profile estimates in traditional assessment conditions, and does incorporating process data improve predictive power? We develop a class of deep-learning CDMs based on the fuzzy CDM framework, supporting conjunctive, disjunctive, and compensatory condensation rules, and extend them to incorporate process data, including response times and action sequences from computer log files, via feedforward and LSTM encoders. Models are evaluated on two large-scale assessments, PIAAC and PISA, using cross-validation for predictive accuracy and pairwise profile agreement for attribute estimation. Deep-learning CDMs showed small but consistent underperformance relative to psychometric CDMs without process data, suggesting that previously reported advantages reflect the structural properties of online tutoring datasets rather than fundamental superiority of neural architectures. Incorporating process data consistently improved deep-learning CDMs, with gains varying by dataset depending on the richness of action sequences. Attribute profile estimates were highly stable across process-data conditions and governed primarily by the assumed response function rather than the estimation method.