Reducing Anxiety and Enhancing Performance: The Impact of AI Chatbots versus Human Facilitation on EFL Speaking Assessment Outcomes

Background Foreign language speaking anxiety significantly impairs learners' ability to demonstrate their true communicative competence, particularly in high-stakes assessment contexts. Artificial intelligence chatbots present a potentially transformative approach to addressing this challenge by providing alternative assessment environments that may reduce affective barriers to performance. Methods This mixed-methods study investigated the comparative impact of AI chatbots and human facilitation on foreign language speaking anxiety and achievement scores among 48 first-year English Language Teaching students at a Turkish public university. Using a within-subjects crossover design, participants completed two comparable speaking examinations—one facilitated by an AI chatbot and the other by human instructors—with Foreign Language Speaking Anxiety Scale assessments preceding each exam. Additionally, students completed a digital literacy assessment to determine technological proficiency levels. Semi-structured interviews with 11 participants provided qualitative insights into student experiences. Results Results revealed a significant negative correlation between speaking anxiety and achievement scores in human-facilitated examinations (r = -0.500, p < 0.01), but no significant correlation in AI-facilitated examinations (r = -0.042, p > 0.01), suggesting AI's potential in mitigating anxiety effects. While achievement scores did not significantly differ between test conditions, students with higher digital literacy performed better in AI-facilitated examinations (r = 0.353, p < 0.05). Thematic analysis of qualitative data revealed four major themes: self-perceived speaking proficiency, comparative perceptions of AI versus human evaluation, intentions for future classroom implementation, and feedback quality assessment. Conclusions The findings contribute to the theoretical understanding of technology-mediated assessment in foreign language education and offer practical implications for creating anxiety-reduced assessment environments. AI chatbots may provide more equitable assessment experiences, particularly for highly anxious learners, though human facilitation remains preferred for its empathetic qualities.