From "Crutch" to Coach: Patterns of Sustained Engagement and Deepening Support in AI Wellbeing Coaching

Background: As use of on-demand, user-led AI chatbots for mental health support grows, a central question is whether these systems provide only short-term assistance or can foster sustained, coaching-style relationships designed to enable lasting change and wellbeing improvements. We examined this distinction in Nova, an AI wellbeing coach. Methods: We analysed 14,293 sessions between January and August 2025. Session-to-session continuity was assessed with large language model (LLM) classification of whether returning users revisited prior issues or initiated new topics. Emotional engagement depth was measured using the Experiencing Scale (EXP), with ratings generated by an LLM calibrated against expert psychotherapists. Sessions were classified by topic using an LLM applied to a taxonomy developed by clinical psychologists. Results: Returning users continued prior work in 70.8% of sessions, indicating sustained engagement rather than episodic support. Regular users (≥5 sessions) achieved deep emotional engagement (EXP ≥5) at more than twice the rate of single-session users (6.5% vs. 2.9%), though users with initially lower engagement scores were more likely to return. Continuity varied by topic: wellbeing- and work-related challenges were frequently revisited, whereas more complex concerns were typically addressed in single sessions, reflecting appropriate boundaries. Calibration of EXP ratings against expert raters yielded moderate to good reliability (ICC = 0.62-0.79). Conclusions: Users not only seek but also develop sustained, coaching-style relationships with AI, within appropriate limits of scope. The convergence of topic continuity and deepening emotional engagement suggests that AI can support iterative coaching processes rather than providing only short-term relief or information. Our approach offers a framework for assessing relationship quality in AI tools for wellbeing.