Efficacy Of Chatgpt-4 in Oral Hygiene Counseling: Evaluation of Information Accuracy and Repeatability

Background Oral hygiene habits play a crucial role in preventing periodontal diseases. Patients often require expert guidance to obtain personal dental tools and oral health information. This study aims to evaluate the performance of ChatGPT-4 in providing high-quality information on oral hygiene habits. Methods Thirty open-ended questions were prepared and categorized under three groups related to maintaining periodontal health. In March 2024, each question was asked of ChatGPT 30 times, resulting in 900 responses. The responses obtained were independently evaluated by experts using a 3-point Likert scale. Three reference sources were used during the question preparation and response evaluation phases. The expert scores' absolute (n) and relative frequency (%) were calculated. Accuracy was assessed using the Wald binomial method, while accuracy across groups was evaluated using the Fisher-Freeman-Halton test. The Benchmark scale assessment was used to determine repeatability and consistency. The confidence interval was set at 95%. Data were analyzed using IBM® SPSS® and R software. Results ChatGPT-4’s accuracy rate in answering the questions was 81.4%. The confidence interval ranged between 78.8% and 83.9%. ChatGPT-4 provided a higher accuracy rate (88.5%) for questions in the "Oral Antimicrobials and Toothpaste" group compared to other groups. Repeatability ranged between "Almost Perfect" and "Substantial." Conclusions ChatGPT-4 could be a valuable virtual assistant integrated into clinics, providing expert-level information on oral hygiene habits for patients and healthcare professionals.