A Framework for Evidence-Based Psychotherapy with AI (EBP-AI)

Large language models (LLMs) offer significant potential to augment or overhaul aspects of psychological assessment and treatment. However, current LLM technologies have yet to demonstrate the capacity to effect meaningful and sustained clinical change. This gap reflects both the limited integration of clinical science knowledge into foundation LLMs and applications built using them, as well as the mismatch between the brief, minutes-long nature of most LLM interactions and the months-long nature of most evidence-based treatments. A principled set of design criteria for developing effective clinical LLM applications are needed to fill this gap. Here we introduce the Evidence-Based Psychotherapy with AI (EBP-AI) framework, which outlines crucial components of evidence-based psychological practice for integration into clinical LLMs: 1) psychodiagnostic assessment, 2) longitudinal case conceptualization, 3) appropriately dosed intervention planning, 4) meaningful progress evaluation, 5) attention to real world implementation and use, 6) clinically appropriate style, and 7) integration of emerging understandings of mechanisms. We introduce a set of key technical questions and considerations for development and evaluation of clinical LLMs against these guidelines. Despite their potential, current LLMs face limitations in meeting these guidelines, including issues with memory, sycophancy, and prioritizing short-term helpfulness over long-term clinical impact. Designing effective, clinical-science-based LLM systems requires understanding the limitations and strategically extending the abilities of LLMs.