Development and Reliability of an Objective Structured Clinical Examination for Intake Interviews in Mental Health Occupational Therapy Education
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Introduction
Evaluating clinical competencies in mental health occupational therapy is hampered by the lack of standardized assessment tools. Intake interviews are particularly difficult to assess because of their interpersonal and observational nature. This study developed an Objective Structured Clinical Examination (OSCE) for intake interviews in mental health occupational therapy education and examined its reliability.
Methods
In this cross-sectional study, 60 third-year occupational therapy students from two Japanese universities each completed one OSCE station using one of three standardized patient scenarios based on common psychiatric presentations. Performance was rated on a 16-item, behaviorally anchored scale across three domains (attitudes, skills, evaluation) by two occupational therapists. Internal consistency was assessed using Cronbach’s alpha, and inter-rater reliability using intraclass correlation coefficients; agreement on total scores was examined with Bland–Altman analysis.
Results
Internal consistency was high for all domains (Cronbach’s α = 0.850–0.887) and for the total scores (α = 0.816 and 0.817). Inter-rater reliability ranged from moderate to excellent across domains (ICC = 0.670–0.900) and was substantial for the total score (ICC = 0.854, p < 0.001). Bland–Altman analysis showed a small mean difference between raters and narrow 95% limits of agreement.
Discussion
This OSCE showed high internal consistency and acceptable inter-rater agreement in assessing complex competencies such as empathy and professionalism. By translating abstract interpersonal skills into observable behaviors, it may provide a practical framework for structured formative feedback. Further studies with larger and more diverse samples are needed to confirm its validity and applicability.
