A Large Language Model Based Teaching Pathway Is Feasible for Developing Clinical Pharmacy Practice Competencies
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background Pharmaceutical care competencies, including pharmacist-patient communication and pharmacy rounds, are core capabilities for clinical pharmacists. However, early-career trainees often lack sufficient real-patient exposure, standardized mentorship, and immediate feedback during skill development. Traditional standardized patients (SPs) are costly and limited in scalability, while early virtual patients (VPs) suffered from rigid, pre-programmed interactions. Large language models (LLMs) offer new opportunities for creating interactive virtual standardized patients (VSPs). However, no study has yet applied an LLM-based VSP teaching pathway specifically to address this early-career training gap in clinical pharmacy. Objective This study developed an LLM-based virtual standardized patient teaching pathway – implemented with the open-source DeepSeek-R1 model – designed to provide standardized, replicable communication practice for early-stage clinical pharmacy trainees. We aimed to preliminarily evaluate its feasibility, score consistency, and user acceptance. Methods The teaching pathway was built using structured prompt frameworks: the SPADE framework for dialogue generation and an RTF (Role-Task-Format)-based prompt for automated scoring. Three standardized oncology pharmacy ward round cases were created. A total of 21 clinical pharmacy trainees (6 junior pharmacists, 15 graduate students) participated in 30 simulation interactions. System performance was assessed via expert panel evaluation (15-item scale), trainee user experience (18-item scale plus open-ended questions), and consistency analysis between automated scores and expert human scores (Spearman correlation, Bland-Altman). Results Experts rated anthropomorphism highest (mean 4.44/5 for natural patient-like speech). The VSP automated scores demonstrated an extremely strong rank correlation with expert scores (Spearman’s ρ = 0.934, p < 0.001), but showed systematic underestimation (mean difference − 2.38 points, 95% LoA − 8.17 to 3.40). System stability received relatively low ratings (3.62/5 for real-time response). Trainees reported high overall acceptance across dimensions (81–89% of maximum scores), with the highest rating for “comfortable without tension” (4.52/5). Qualitative feedback highlighted realistic details and reduced anxiety, while noting occasional over-answering and technical delays during peak hours. Conclusions The LLM-based VSP teaching pathway, guided by SPADE and RTF-based prompt frameworks, can simulate authentic pharmacy ward round dialogues and generate scores that strongly correlate with expert judgments. Although systematic underestimation precludes its use for high-stakes assessment, the system shows strong promise as a low-cost, accessible formative training tool for pharmaceutical care. The prompt engineering frameworks are transferable to other LLMs and clinical communication scenarios.