An Adaptive Smart Architecture Using Real-Time Multimodal Learning Analytics and Deep Reinforcement Learning

Personalised learning at scale is one of the core challenges in intelligent tutoring systems (ITS) mainly because learners states are high dimensional, learning modalities are heterogeneous, and learner behaviour changes over time. The article presents the Adaptive Smart Architecture of Real-Time Multimodal Learning (ASARTML), a new framework combines real-time multimodal learning analytics and deep reinforcement learning (DRL) engine to provide dynamically responsive and context-sensitive pedagogical content. ASARTML combines six data streams, including clickstream logs, eye-tracking cues, electroencephalography (EEG), scoring on tests, facial action unit in videos, and natural language interaction based on a cross-attention multimodal fusion network. The policy backbone is a dueling deep Q-network with a long short-term memory (LSTM) module. The architecture also includes a Bayesian reward model, which integrates instant feedback of performance and long-term alignment of learning through the knowledge graph. Empirical analysis of 1,240 participants shows that ASARTML gives a classification accuracy of 87.3% in identifying state of learners (F1 = 0.869), statistically significant average learning of 34.8% in comparison with a static curriculum control (Cohen d = 1.84, p = 0.0001). These findings highlight why multimodal DRL systems can significantly expand adaptive learning on an institutional level. Clinical Trial Registration: ClinicalTrials.gov, identifier NCT06284721 ; registered 15 February 2024.