Facial Human Analysis for AI-Driven Interviews

Maintaining fair, consistent, and equitable applicant assessments has become a major difficulty as virtual interviews and evaluations grow more widespread. In light of this, we suggest implementing a real-time facial emotion recognition software solution that incorporates both computer vision and deep learning techniques for developing a measurement of candidate confidence throughout interviews conducted by AI. Utilizing OpenCV’s Haar Cascade for face detection and training a Convolutional Neural Network (CNN) using FER-2013 dataset to identify five different facial expressions (Happy, Neutral, Sad, Angry, and Surprise), a Confidence Estimation Module allows this emotional data to be correlated with distinct sections of an interview and presents useful behavioural insight. In trials, the system operates in real-time and has demonstrated delays from 60–80 milliseconds and an acceptable correlation coefficient value (0.71) between the system's confidence metrics and user-reported confidence measurements. As such this system represents a valuable method for improving the overall fairness of virtual evaluations while retaining human oversight and adhering to ethical practices.