A critical appraisal tool for evaluations: learning from Evidence Based Medicine

Evaluation may be facilitated by the application of tools derived from the practice of Evidence Based Medicine (EBM). This paper presents a proposed framework and critical appraisal tool for assessing publications and evaluation-related reports. There are two components intrinsic to the framework - the pragmatic measures or practical steps involved in conducting an evaluation and the requisite constructs that support these steps. These steps include: defining the purpose of the evaluation; measuring and understanding both process and outcome data; and learning from the evaluation. Besides, it is important to revisit the evaluation in order to extract valuable insights and modify the current as well as any future evaluation as an iterative process. Underlying such practical steps are various constructs that include theory, methods, values, dynamics of complexity, communication, reflection, and context. By taking into account both the practical steps and the underpinning constructs, we can build capacity through ‘learning by doing’. This approach enables education to take place in the context of evaluation in practice, as we follow the steps and gain knowledge regarding the underpinning constructs. A checklist has been provided to facilitate the critical appraisal of publications and evaluation reports in accordance with the framework. The critical appraisal checklist was implemented for assessing the published evaluation of a global public health capacity building programme. The checklist was determined to be both relevant and user-friendly for this specific evaluation. The paper is presented with the expectation that the framework and critical appraisal checklist will be beneficial in the appraisal of publications and evaluation reports, as well as in the development of educational programmes in order to promote global capacity in the context of evaluation.