The Changing Landscape of Validity-Centered Cognitive Learning-Instructional-Assessment Design Frameworks

Assessment design is a topic that is not only of interest for instructors but also for cognitive psychologists and learning scientists, as they are the tools that make the objective and reliable measurement of latent constructs possible. Yet, the design of such assessments still remains an art, especially for naturalistic settings like classroom assessments. Most of the developments in psychometrics and assessment design frameworks have been aimed at large-scale, high-stakes testing. In this paper, we argue why such tools may be inadequate for classroom assessments and then explore some of the ways we can extend these frameworks for the classroom setting. The frameworks were chosen based on some specific criteria such as: the degree to which cognitive theories of learning and task performance are integrated into the design process, the emphasis on construct validity and measurement models, and the flexibility of the design framework with existing measurement modeling frameworks. We also use an example of a reading comprehension task to compare how many of the assessment design frameworks are similar and different at the same time. A theme of assessment design frameworks evolving to accommodate the ever-changing needs of educational measurement also emerged as we did this review.