The Burden of Autonomy: Ethical Concerns Among Active Therapeutic Users of Generative AI for Emotional and Mental Health Support

Introduction: The rise of generative artificial intelligence (GenAI) has provided opportunities for the general public to use for emotional and mental health support (EMS), despite the absence of regulatory oversight for most uses. While ethical concerns have been raised by experts in mental health and ethics, little is known regarding how users with lived experiences of using GenAI for mental health may perceive the risks. In this paper, we examined the ethical concerns of active therapeutic users of GenAI for EMS. Methods: We screened 5,386 individuals online globally concerning the use of a specific AI (ChatGPT 3.5 or 4), and utilized responses from 270 users, from 29 countries, who repeatedly used ChatGPT for EMS, on their concerns regarding using GenAI for EMS.Results: We discovered four themes that capture users’ ethical concerns: (1) Information Accuracy, (2) Competency, Emotional Awareness, and Liability, (3) Unfair Influences, and (4) Data Protection. We then contrasted our themes with the ethical principles in the APA guidelines to analyze similarities.Conclusion: Our findings revealed the burden of autonomy, where the ethical responsibility for quality and safety of mental health support has been shifted from human professional providers to users themselves when seeking self-help from GenAI for EMS. The results highlighted users’ ethical awareness and underscored the need for developers and regulators to implement robust ethical guidelines that bridge the gap between AI technology and traditional mental health standards to prevent psychological harm.Keywords: GenAI, ethics, qualitative analysis, user perspective, emotional and mental health support