Retrieval-Augmented Generation in LLMs for Mental Health: Examining the Impact on User Intent Detection in Wysa

Background Digital mental health interventions (DMHIs) offer scalable support but ensuring they accurately detect users’ intent during volatile situations can be challenging. Pure parametric Large Language models (LLMs) do not contain specific safety critical architecture, and can miss critical cues, or hallucinate, undermining reliability. Retrieval Augmented Generation (RAG), which supplements an LLM with retrieved context, could enhance intent detection during volatile situations. This study examined whether integrating RAG into a specific DMHI (Wysa) improves intent classification and safety risk detection compared to using the base LLM alone. Methods This paper evaluates six LLM models within Wysa via a controlled comparison of RAG-enabled versus RAG-disabled modes. Anonymized real and synthetic user-chatbot exchanges were manually labeled against multi-class intent categories (e.g. self-harm, abuse, panic). We computed classification accuracy, recall, precision and F1 scores against ground truth labels and tested differences for statistical significance. Performance was also examined by risk category and inter-model agreement. Results RAG consistently improved safety intent detection with five of six models demonstrating significant accuracy gains, especially with smaller models (with increased accuracy ranging from ~ 48% to ~ 73% with RAG). Recall for high-risk intent (like child abuse, and panic attack) increased substantially, with missed cases dropping by over half. Models also normalized to reach consistent agreement predictions under RAG. However, improved sensitivity came with a slight increase in false negatives, reflecting a precision-recall trade-off (fewer instances of missed risk at the cost of more false alarms). Conclusions Integrating RAG in mental health settings enhances detection of user intent and safety concerns. RAG particularly benefited smaller LLMs, narrowing performance gaps with larger, advanced models and reducing missed critical flags. While RAG caused a rise in false alarms, the trade-off is acceptable in a safety-critical context. Overall, these findings support RAG as a promising approach to improve the accuracy, consistency and safety of LLM-driven DMHIs.