Diverse approaches to sentiment analysis reliably reflect and explain symptom changes in psychotherapy

Psychopathology is often characterized by intense negative emotions. Understanding how these emotions are expressed through language can provide valuable insights on emotional processing and reveal linguistic patterns associated with mental health conditions. Sentiment, or the emotional tone of language, has been linked to affective states and wellbeing, suggesting it could serve as a marker of psychopathology and therapeutic progress. However, the versatility of different approaches to measuring sentiment makes it unclear which methods are best, and they have not yet been deployed in a longitudinal therapeutic setting. This study examined whether changes in client sentiment, therapist sentiment, or their divergence tracked clients' internalizing symptoms using a large dataset of text-based psychotherapeutic exchanges. We compared 11 commonly used sentiment analysis measures across exploratory (N=3,729) and validation (N=2,500) datasets. Both client and therapist sentiment became more positive over time, mediating the link between time in therapy and decreased internalizing symptoms. Client and therapist sentiment diverged over time, with therapists becoming more positive than clients, and on months when therapists were more positive than clients, clients reported fewer internalizing symptoms. Diverse sentiment measures mostly behaved similarly, although approaches that estimate word-level sentiment and treat positivity and negativity as separate constructs tended to yield more consistent and nuanced results. These findings suggest that language sentiment can be a marker of internalizing symptoms and therapeutic progress, highlighting the potential of sentiment analysis as a tool for monitoring treatment outcomes.