Can Artificial Intelligence Evaluate Online Health Information? A Comparative Assessment of Scabies-Related YouTube Videos Using Human Experts and ChatGPT-5

Scabies is a highly contagious parasitic skin disease for which patients frequently seek information on YouTube, although the reliability of available content is uncertain. Artificial intelligence (AI), particularly large language models, has emerged as a potential tool for assessing online health information; however, its concordance with expert evaluation remains unclear. This cross-sectional study analyzed the first 50 English-language YouTube videos retrieved using the term “scabies disease.” Two dermatology specialists independently evaluated videos using the DISCERN instrument, JAMA benchmark criteria, and the Global Quality Scale (GQS). Video characteristics were recorded, and sources were classified as professional or non-professional. Corrected transcripts were analyzed with ChatGPT-5 to generate Accuracy and Completeness scores. Readability was assessed using Flesch Reading Ease and Flesch–Kincaid Grade Level. Professionally produced videos scored significantly higher than non-professional videos across all human-based quality measures (p < 0.01). AI-generated scores were also higher for professional content but showed only moderate correlation with expert assessments. ChatGPT Completeness demonstrated moderate discrimination in identifying higher-quality videos (AUC = 0.668). Overall, AI reflected general quality trends but did not replicate expert judgment, suggesting a complementary rather than substitutive role.