"A Safe First Step": Design and Evaluation of an Emotionally Expressive AI Virtual Patient for Clinical Simulation in Speech-Language Pathology Training

Speech-language pathology (SLP) training programs face persistent challenges in providing sufficient opportunities for clinical placements or simulations that allow students to practice clinical communication with patients with communication disorders. We designed and implemented V.O.I.C.E., an AI-powered virtual patient system that enables realistic, open-ended clinical simulation for SLP students. V.O.I.C.E. integrates (1) a large language model (LLM) role-play agent designed to produce responses consistent with symptoms of post-stroke expressive aphasia (Broca’s aphasia), (2) a multimodal emotional expression pipeline that renders context-appropriate emotions of the virtual patient through coordinated speech prosody, facial expressions, and gestures, and (3) an LLM-based, rubric-guided debriefing module that generates structured formative feedback across core clinical communication competencies. We conducted a pilot mixed-methods, quasi-experimental evaluation with 11 graduate-level SLP students, where they completed three simulation sessions using the V.O.I.C.E system with progressive challenges. Results show significant gains in students' overall self-efficacy for interacting with Broca's aphasia patients in clinical settings. The self-efficacy gains were moderated by students’ perceptions of multimodal realism of the simulations. Qualitative findings highlight the system’s value as a low-stakes “bridge” for transition to clinical practice, which supports learning through iterative practice, reflection, and strategy refinement. This work provides a framework for AI-based simulation training applications that emphasizes interpersonal communication.