The Scope and Limitations of Extant Research into ChatGPT as a Tool for Patient Education: Systematic Review
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Chat Generative Pre-Trained Transformer (ChatGPT), a large language model (LLM) developed by OpenAI, has been extensively studied and embraced by medical researchers since its public release in November 2022. In addition to its benefits in generating summaries and predictive diagnostics, it has been proposed as a patient education tool. Existing research cites ChatGPT’s potential to increase information availability and accessibility, improve the efficiency of clinical practice, and its high-quality responses to clinical questions as reasons to consider its adoption.
Objective
We assessed literature from PubMed on the quality and consistency of ChatGPT responses across medical specialties, evaluated the comprehensiveness of current research, and identified areas for future research.
Methods
We searched PubMed for published articles evaluating the consistency, reliability, and ethics of ChatGPT in patient education. Following the PRISMA guideline, we conducted a systematic literature review of the 567 retrieved records. After title, abstract, and full-text screens, 123 relevant records were included and synthesized in this review.
Results
We found a lack of consensus among ChatGPT studies. The model accuracy suffers from infrequent updates and generation of misleading information (hallucinations), and it lacks knowledge of current clinical guidelines. The consistency of the model falls short due to its sensitivity to prompt design and fine-tuning through user interaction, making ChatGPT research results almost impossible to peer review and validate. Relying on ChatGPT for clinical information risks spreading misinformation, disrupting trust in the medical system, and disobeying the principles of patient-centered care. This also shifts the focus of patient education from shared decision-making and information-building to information-giving, deviating from the objectives of Health Communication and Health Literacy outlined in Healthy People 2030.
Conclusions
We caution against the acceptance of ChatGPT as a patient education tool and encourage future efforts to incorporate community perspectives in AI research.
