Zero-Shot Evaluation of Kimi K2 on Pediatric Clinical Cases
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
The application of large language models (LLMs) in pediatric medicine requires rigorous performance evaluation prior to clinical implementation.
Objective
To evaluate the accuracy of the Kimi K2 model in analyzing pediatric clinical cases using a zero-shot approach. Methods: 2,249 multiple-choice questions from pediatric clinical cases, ranging in age from 1 day to 16 years, extracted from the MedQA dataset were analyzed. The model was tested via API with standardized parameters, temperature set to zero, and zero-shot prompts. Accuracy was calculated by comparing the responses with the dataset’s ground truth.
Results
Kimi K2 achieved an overall accuracy of 78.39%, corresponding to 1,763 correct answers out of 2,249 total, with 100% of responses in the required format. Conclusions: The model demonstrates competitive performance for medical education and diagnostic support, while still having limitations that require human clinical supervision.
