A Review of the Reliability and Validity of Objective Structured Clinical Examinations (OSCE) in Assessing Clinical Competency in Anaesthesia: A Systematic Review

The Objective Structured Clinical Examination (OSCE) has become a cornerstone in assessing clinical competence in medical education, particularly in anaesthesia training and certification exams. This systematic review evaluates the reliability and validity of OSCEs in anaesthesia assessments, addressing the critical question of whether they effectively measure clinical competency for consequential decisions such as licensure and certification.A comprehensive literature search was conducted using databases including PubMed, Google Scholar, Embase, ERIC, and Scopus, adhering to PRISMA guidelines. Studies published between 2000 and 2024 were included, focusing on anaesthesia trainees or practitioners undergoing assessments. Key outcomes examined were reliability metrics (internal consistency, inter-rater reliability) and validity evidence (content, construct, criterion, and consequential validity).Findings indicate that OSCEs demonstrate strong reliability when properly designed, with Cronbach’s alpha values often exceeding 0.7 and inter-rater reliability ranging from 0.7 to 0.9. Validity evidence supports OSCEs as robust tools for assessing clinical skills, particularly when scenarios align with real-world clinical tasks and scoring rubrics are standardised. However, challenges such as examiner bias, station heterogeneity, and resource intensity were identified as limitations. Innovations like virtual OSCEs (VOSCEs) and simulation-based assessments show promise but require further validation.The review concludes that OSCEs, when implemented with rigorous examiner training, standardized scenarios, and iterative quality improvements, offer a reliable and valid method for evaluating clinical competence in anaesthesia. Recommendations include enhancing examiner calibration, integrating workplace-based assessments, and exploring technological advancements to address current limitations. Future research should focus on multi-institutional validation and longitudinal studies to strengthen the generalizability of OSCE outcomes.