Use of artificial intelligence-based facial expression recognition for diagnosis and therapy monitoring in patients with post-paralytic facial nerve syndrome

Artificial intelligence (AI) based technologies are already frequently used for facial expression recognition (FER) in healthy controls. The question is if these AI-based techniques can also be used for objective classification and therapy monitoring of patients with facial palsy. This could finally help to completely replace the subjective grading systems. 36 patients with post-paralytic facial nerve syndrome synkinesis (PFS) were analyzed before and after 9 days of facial biofeedback training. 36 healthy controls were analyzed in the same time interval. 3D videos were recorded while all participants imitated the emotional expression “happiness.” Three synthetic images were created: 1) contralateral side (C) on the right side and synkinetic side (S) on the left side (image CS), 2) S and flipped S (image SS), 3) C and flipped C (image CC). The FER classifier Residual Masking Network (RMN) was used to estimate “happiness” in CS, SS, and CC images. RMN output was correlated with clinical parameters such as facial nerve grading systems or patient-reported outcome measures. RMN could analyze both facial sides in CS, SS, and CC images separately with an expression of the emotion “happiness” and distinguish differences between patients and controls. RMN could be used for the diagnosis of changed emotional expression in patients with PFS.