The (not-so) valid and reliable linguistic markers of depression and anxiety in symptomatic adults: A randomised cross over trial

Psycho-linguistic theory postulates that linguistic features expressed in individuals’ communications may be valid indicators of their mental health. This trial aimed to examine the validity and reliability of linguistic markers of depressive and anxiety symptoms in adults. Using a randomised cross over trial design, 218 adults provided eight different types of text data of varying frequencies and emotional valance including SMS data, social media posts, personal expressive essays, and letters to a friend. Linguistic features were extracted from each task using LIWC22 software and correlated with participants’ self-report symptom scores. Machine learning models were used to determine which linguistic features had the strongest associations with symptoms. There were no linguistic features consistently associated with depressive or anxiety symptoms within tasks or across all tasks. Features found to be associated with depressive symptoms were different for each task and there was only some degree of reliability of these features within the repeated tasks. In all the machine learning models, predicted values were weakly associated with actual values for both depressive and anxiety symptoms. Some of the text tasks were found to have lower levels of engagement and negative impacts on participants’ mood. Overall, these findings indicate that there may be few valid and reliable group-level linguistic markers of depression and anxiety when examining several types of individuals’ text data.