Evaluating a Medical-Grade Voice AI for Patient and Caregiver Guidance: A Multi-Scenario Nurse Panel Study

Medical-grade conversational AI offers the potential to extend patient support programs (PSPs) in regulated therapeutic areas, but its safety and reliability must be rigorously evaluated. We conducted a large-scale, nurse-led assessment of a voice-based AI system across 30 patient and caregiver scenarios spanning diabetes, oncology, neurology, cardiometabolic disease, and rare disorders. Nearly 1,000 U.S.-licensed nurses role-played patients or caregivers in more than 20,000 interactions, scoring the AI across five domains: clinical accuracy, empathy, communication clarity, appropriateness of advice, and compliance with evidence. The system achieved over 97% top ratings across all domains, with empathy noted most strongly in oncology and rare disease caregiving contexts, and clarity reflecting minor opportunities in pacing and call-closing behavior. Qualitative feedback emphasized tone, personalization, and regulatory compliance as consistent strengths. These findings demonstrate that voice-based AI can safely and effectively support patient and caregiver interactions, suggesting readiness for scaled deployment in pharma-led PSPs to expand after-hours coverage, provide consistent patient engagement, and generate feedback loops to inform both AI and human nurse training.