AI-assisted modeling of attention quantifies engagement and predicts cognitive improvement in older adults
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Cognitive training aims to prevent or slow cognitive decline in older adults, but outcomes vary widely. Engagement, describing how individuals allocate cognitive, affective, and physiological resources, is critical to training benefits, yet behavioral metrics lack real-time modeling of attention and do not reliably predict outcomes. We developed and validated a multimodal, AI-assisted biomarker that quantifies attentional states during computerized cognitive training and predicts cognitive improvements.
Methods
We designed the Attentional Index using Digital measures (AID), leveraging a video-based facial expression encoder (pretrained on 38,935 videos), an ECG-based autonomic encoder (pretrained on 123,998 ECG samples), and a temporal fusion module. Using two processing speed/attention studies in older adults (> 65 years) with mild cognitive impairment, AID was trained and evaluated in BREATHE (n=50; ∼300 hours from 368 sessions) and validated in FACE (n=20; ∼150 hours from 219 sessions). Model training targeted session-level change in self-reported fatigue. Clinical validation tested relationships between AID scores and (1) behavioral attention, (2) cognitive outcomes, and (3) neural correlates.
Findings
AID accurately detected engagement changes (BREATHE: accuracy 0.82, F1=0.81; FACE: accuracy 0.73, F1=0.74), outperforming unimodal models. AID scores were unrelated to session, task type, or demographics. In BREATHE, session-level AID scores significantly predicted executive function improvement (session×AID: Wald χ 2 =7.85, p=0.005), whereas reaction time variability did not. Lower AID intercepts (B=-0.07±0.03, p=0.043) and steeper slopes (B=0.31±0.15, p=0.046) were associated with greater improvements. Post hoc analyses identified two engagement profiles linked to better attention: one characterized by low-RMSSD and focused periocular activation, and the other defined by coherent alignment between low-RMSSD and facial expression patterns.
Interpretation
AID provides a reliable digital biomarker of effective engagement and predicts cognitive improvement beyond behavioral metrics. By capturing facial-autonomic dynamics of attention, AID offers a foundation for closed-loop cognitive intervention design.
Funding
NIH AG081723, NR015452, and AG084471; Stanford HAI seed funding.
Research in context
Evidence before this study
We searched PubMed, Google Scholar, Web of Science, IEEE Xplore, and reference lists of relevant reviews for studies published in English before March 1, 2025. Search terms included combinations of “cognitive training,” “engagement,” “attention,” “older adults,” “mild cognitive impairment,” “heart rate variability,” “facial expression,” “psychophysiology,” “multimodal,” “machine learning,” and “digital biomarkers.” We included studies involving older adults, computerized cognitive training (CCT), and behavioral markers of engagement/attention. We excluded studies that did not involve aging populations, did not report attention or engagement measures, or lack objective psychophysiological or behavioral signals.
Across the evidence base, behavioral performance metrics (reaction time, accuracy) showed inconsistent associations with training-related cognitive outcomes and lacked the capacity to capture attentional dynamics. Psychophysiological markers reflected arousal or effort but were rarely linked to cognitive transfer effects. Importantly, no study identified through our search evaluated a real-time, multimodal framework combining facial and autonomic signals to quantify attention during CCT, nor did any study provide external validation across independent cohorts. The quality of existing evidence was moderate, with common limitations including small samples, limited generalizability, and high risk of bias due to variations in self-reports.
Added value of this study
This study establishes, for the first time, a multimodal, AI-assisted biomarker that quantifies attentional states during CCT by integrating facial and autonomic features. The AID framework was comprehensively validated, demonstrating reliable performance across two independent aging cohorts and robustness across tasks and sessions, and more accurately predicted cognitive improvement than conventional behavioral metrics. Our findings introduce a clinically interpretable engagement biomarker that correlates with neural and psychophysiological signatures of attention, overcoming limitations of behavioral-only approaches.
Implications of all the available evidence
In summary, prior evidence and our findings suggest that multimodal measures integrating facial and autonomic signals may provide a more detailed description of effective engagement during cognitive training by modeling both the attentional availability and allocation. Such measures could eventually help refine non-pharmacological interventions for older adults at risk for cognitive decline and inform future research in personalized, closed-loop cognitive training design. However, although AID shows promise as an objective and generalizable indicator of attentional state, further validation in larger and more diverse samples is required. At this stage, AID should be regarded as a tool that contributes to understanding how attentional dynamics relate to training response.
