An Advanced Approach to Emotion Detection for Enhancing Online Learning Environments

When it comes to education today, online learning tools significantly impact how students learn. Understanding and evaluating students' behavior can get them much more involved and help them learn more. This paper presents a real-time emotion detection system that can classify emotions into three main categories: engagement, confusion, and boredom. The system utilizes the DISFA dataset, which consists of video recordings of 26 subjects (each four minutes long) annotated with 12 Action Units (AUs). The data set is then preprocessed and converted into frames using FFmpeg, from which unique frames are extracted using DupeGuru. A convolutional neural network (CNN) model, created with Keras and TensorFlow, is trained to detect these emotions accurately, with a classification accuracy of 92.92\% , surpassing the base model. GAN-based data augmentation was also applied to generate additional bored emotion samples, enhancing the dataset and boosting CNN accuracy to 93.46\%. The system has an intuitive web interface designed with Django and integrates the Agora SDK to facilitate real-time video conferencing in virtual classes. The proposed system also offers real-time feedback on student emotions, enabling teachers to obtain important information to adapt and deliver a more effective, responsive education. The running source code is available at: https://github.com/mahiraaly/Real-Time-Emotion-Detection-Using-CNN.