Towards Understanding Bipolar Disorder Through Social Media and Transformer Models: Challenges and Insights

Social media presents a promising avenue for monitoring mental health, yet detecting bipolar disorder (BD) remains significantly underexplored. The complexity arises from the overlap of linguistic patterns associated with depression and anxiety, making accurate identification challenging. This study aims to benchmark the performance of various transformer-based large language models (LLMs) trained on Reddit posts, to distinguish BD from other mental health conditions. Using a high-performing LLM (GPT-4o) as a benchmark, the analysis reveals that certain fine-tuned open small models (ex. MISTRAL, LLAMA) excel in capturing subtle linguistic cues linked to BD, achieving an F1 score of up to 0.86 with high precision and recall. However, BD was frequently misclassified as depression (range: 23%,51%), normal (range: 2%, 41%), and anxiety (range:1%,7%), underscoring the need for improved approaches. Integrating large language model insights with clinical expertise could significantly enhance bipolar disorder diagnosis and management.