Smoke and LLMirrors: Against the Use of Generative AI in Mental Health Care
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
With little concern for socially deleterious outcomes, companies selling generative artificial intelligence (GAI; ChatGPT, Gemini, Copilot, Claude, etc.) have begun infiltrating the mental health space. We argue that profit-driven forces are taking advantage of overworked mental health professionals via technologies that do not address cracks in the foundations of our healthcare systems and, instead, cause more harm than good. Throughout this manuscript, we aim to accomplish a tripartite goal: a) highlight the market forces promoting AI adoption, b) challenge the illusion of control in AI usage, and c) elaborate on the oxymoronic nature of ‘human-centered’ AI within the mental health fields. Supporting our argument through a comparative investigation are connections between the current GAI industry and another profit-driven industry that made use of clinician collusion to sell the public on a product: Big Tobacco.
