An Integrated Methodology for Assessing Item Discrimination in Mathematics Assessments

In order to assess an assessment, one must consider whether the assessment produces valid and reliable results according to its intended goals. Given the various ways that validity and reliability can be assessed based on the goals of some assignment, there is no singular methodology for analyzing an educational assessment. This study attempts to address this research gap by presenting a unifying methodology that incorporates both CTT and IRT, along with other measures of validity and reliability, that can provide a robust analysis of an educational assessment while considering the goals of that assessment. Utilizing item measures from Classic Test Theory, Item Response Theory, and Distractor Analysis, we present a summary of each metric that will inform labeling items as sufficiently discriminating for the purposes of an educational assessment. We then present an analysis of 4 exams to illustrate how the metrics can be used in concert to identify validity and reliability at the item and assessment level. We conclude with a brief discussion of how this methodology can be applied to different types of assessments.