There’s not an APP for That: Comparing Interpretation Through an AI Voice App and Qualified Medical Interpreters in Real World Clinical Settings

Background/Objectives: AI voice interpretation applications are increasingly used in clinical settings to address language access challenges, yet evidence comparing their performance to qualified in-person medical interpreters in authentic clinical encounters remains limited. This study compares the linguistic and clinical accuracy of AI-based voice interpretation and certified in-person medical interpretation using recorded real-world clinical encounters. Methods: Outpatient clinical encounters involving patients with limited English proficiency were audio-recorded. Fourteen physician speech segments (mean length 78.5 words) representing common diagnostic, treatment, and counseling content were extracted and translated into seven languages using both an AI voice interpretation application and certified in-person medical interpreters. Bilingual reviewers and a clinician evaluated translations for accuracy, completeness, and clinical fidelity. Qualitative analyses examined error patterns and contextual loss; quantitative comparisons assessed error rate differences across languages and interpretation conditions. Results: AI voice app translations exhibited significantly higher linguistic errors (χ2[1] = 19.78, p < .001) and clinical accuracy errors (χ2[1] = 45.07, p < .001) than qualified medical interpreter translations. Clinical error rates were 33.3% for AI-generated versus 4.8% for interpreter-generated translations. Error rates were also higher for less commonly spoken languages compared to commonly spoken languages when using the AI voice app (42.9% vs. 14.3%). Conclusions: Qualified in-person interpreters remain essential for safe, accurate clinical communication. Hybrid models integrating professional interpretation with appropriately deployed AI technology may offer a balanced approach to expanding language access while maintaining communication safety and equity.