Evidence on the validity, reliability and usability of active and passive sensing of cognition for use in ambulatory studies: A systematic scoping review

Cognitive functions are traditionally measured as trait-like constructs in highly controlled lab settings using paper-based or computerized tasks. Growing emphasis on ecological validity and the increasing use of ecological momentary assessment (EMA) have shifted research focus towards repeated or continuous assessment of cognitive states in daily life. This includes cognitive tasks or questions delivered via mobile devices (active sensing), and objective indicators such as keystroke dynamics derived from smartphone sensors (passive sensing). However, the psychometric properties and usability of these approaches have not yet been established systematically. This scoping review evaluated the evidence on validity, reliability and usability of cognitive assessments for use in ambulatory research. In total, 88 studies were included, with most examining active cognitive sensing (n = 72) and fewer studies on passive sensing (n = 13) or both (n = 3). Among active cognitive assessments, processing speed tasks demonstrated the most consistent and robust psychometric evidence on mobile devices relative to other cognitive domains. Keystroke dynamics, among other passive sensing features, showed consistent initial associations with processing speed measures, and some evidence to distinguish between clinical and non-clinical groups. Despite these promising findings, only a few studies explicitly assessed between- and within-person reliability within EMA settings. Usability outcomes were generally positive across studies but less frequently investigated than validity and reliability. Future research should develop and adopt standardized approaches to evaluate mobile-based cognitive assessments. Eventually such measures will enable research on cognitive fluctuations and their dynamic relationships with stress, behavior and health in real-world contexts.