On the Use of Machine Learning Models for Psychometric Inference: Numeric Validation of Kernel PCA and Autoencoders in Multimodal Measurement Data

With the increasing use of computers in educational and psychological assessments, measurement data are now routinely collected in multiple modalities, enabling a detailed account of individual's test-taking performance. While the profusion of data sources enriches the assessment landscape, the intricacy and multiplicity inherent in the multimodal data pose challenges for psychometric inference. In this study, we investigate the potential of machine learning models for deriving trait scores from multimodal measurement data. We consider two classes of unsupervised models, kernel principal component analysis and autoencoders, and examine their empirical performance across various measurement settings. Numerical experiments using simulated and real-assessment data suggest that machine-learned scoring recovers generating trait scores reasonably well and demonstrates close alignment with traditional scoring methods. The findings imply that machine-learned models may offer practical alternatives in situations where conventional methods are less applicable.