Comprehensive Analysis of Knee Joint Mobilization Skills in Physical Therapy Education: Validity and Reliability of On-Site and Off-Site Direct Observation of Procedural Skills (DOPS)
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background: The direct observation of procedural skills for knee joint mobilization (DOPS knee ) tool requires validation to ensure its reliability and validity for providing immediate feedback and assessing its applicability in off-site settings. We aimed to describe the implementation and clinimetric properties of DOPS knee in both on-site and off-site environments, as well as to evaluate the preclinical competencies of physical therapy (PT) students. Methods: Following the development of DOPS knee , 42 preclinical PT students were filmed performing knee joint mobilization. Their performance was assessed by two clinical instructors using DOPS knee , in on-site and off-site conditions, with an interval of 1.5–4.5 months between assessments. Results: DOPS knee exhibited good on-site inter-rater reliability (ICC = 0.78). However, it had poor off-site inter-rater reliability (ICC = 0.37) and intra-rater reliability differed substantially between on- and off-site evaluations (evaluator 1: ICC = 0.42; evaluator 2: ICC = 0.88). Nevertheless, DOPS knee exhibited good reliability for average measures from on- and off-site evaluations (ICC = 0.84). Average total DOPS knee scores were significantly correlated with average global rating scores in both on-site ( r = 0.47) and off-site conditions ( r = 0.75), as well as in cross-over conditions ( r = 0.54 and 0.71). The cutoff score for DOPS knee was determined to be 14 points. Construct validity analysis of both on- and off-site evaluations revealed that students who passed the DOPS knee evaluation had significantly higher average global rating scores ( p < 0.05). Conclusions: DOPS knee exhibited good validity and reliability for assessing student performance in on-site conditions. However, caution is recommended when interpreting off-site results. Using average measures may improve reliability in off-site assessments.
