Exploring Biases Related to the Use of Large Language Models in a Multilingual Depression Corpus

Recent advancements in Large Language Models (LLMs) present promising opportunities for applying these technologies to aid the detection and monitoring of Major Depression Disorder (MDD). However, demographic biases in LLMs may present challenges in the extraction of key information. This study evaluates commonly used LLMs in the speech health literature, across a cohort comprised of English, Spanish, and Dutch speakers with recurrent MDD to observe the effects of different demographic imbalances. Results indicate demographics indeed influence model performance. Gender showed variable impacts across models, with age presenting more pronounced differences. Model performance also varied across language. This study emphasizes the necessity of incorporating demographic-aware models in health-related analyses. It raises awareness of the biases that may affect their application in mental health and suggests further research on methods to mitigate these biases and enhance model generalization.