Inter-rater reliability of CATHIS core: a study protocol

BackgroundThe Critical Appraisal Tool for Homeopathic Intervention Studies (CATHIS) has recently been developed to improve the overall assessment of the quality of evidence in systematic reviews on homeopathy. To align the instrument with current versions for assessing internal validity, we present a shortened version called ‘CATHIS core’ that retains the last three domains (credibility, coherence, clinical relevance). The aim of this study is to evaluate the inter-rater reliability of CATHIS core and identify potential weaknesses in its implementation to optimize its future application.MethodsFour reviewers will evaluate a total of up to 30 controlled studies on homeopathic interventions using CATHIS core. Inter-rater reliability will be calculated using Gwet’s AC1/2 and Fleiss' Kappa.To examine domain relationships we will calculate correlation coefficients, to identify potential barriers to the use of CATHIS core, we will measure the time required to complete the evaluation per study and assess the user experience applying a standardized questionnaire.DiscussionThe assessments of reporting, model and external validity impact the quality of evidence from systematic reviews on homeopathy. This study will improve our understanding of CATHIS core and how to best use it.