Interactive robot with multimodal multitask model for early screening of multiple common adolescent mental disorders

The early detection of mental disorders in adolescents represents a significant global public health challenge. Due to the complex and subtle nature of mental disorders, making it difficult to detect abnormalities using a single factor. Additionally, the generalized multimodal C omputer- A ided S creening ( CAS) systems, incorporating interactive robots for adolescent mental health assessment, remain unavailable. In this study, we present an Android application equipped with mini-games and chat recording, deployed in a portable robot, to screen 3,783 middle school students. This system generates a multimodal screening dataset comprising facial images, physiological signals, voice recordings, and textual transcripts. We develop a model called GAME ( G eneralized Model with A ttention and M ultimodal E mbraceNet) with novel attention mechanism that integrates cross-modal features into the model. GAME evaluates adolescent mental conditions with high accuracy (73.34% – 92.77%) and F1-Score (71.32% – 91.06%) and outperforms traditional methods. Our findings reveal that each modality contributes dynamically to mental disorder detection and the identification of comorbidities across various disorders, supporting the feasibility of an explainable model. This study provides a system capable of acquiring multimodal information and constructs a generalized multimodal integration algorithm with novel attention mechanisms for the early screening of adolescent mental disorders.