Collaborative quality assessment of simulation-based interactive learning environments for climate education

Simulation-based interactive learning environments (ILEs) are widely used in climate education to foster systems understanding and experiential learning. Yet, most evaluations emphasize predefined learning outcomes, offering limited insight into how learners experience these tools and what quality means to them. This study introduces an adapted Collaborative Quality Assessment (Co-QA) framework as a participatory, developmental evaluation approach for simulation-based ILEs. This approach invites users to co-define quality criteria and assess tools against those criteria, enabling a context-sensitive appraisal of instructional design decisions. We applied Co-QA to En-ROADS, a climate policy simulator, across eight workshops with 104 learners. Our mixed-methods design combined qualitative and quantitative assessments structured around five quality principles: salience, accessibility, credibility, legitimacy, and effectiveness for systems understanding. By foregrounding learner perspectives, Co-QA reveals process-based dimensions of quality that conventional outcome metrics may overlook. Our findings indicate that the ILE supports broad exploration of climate policies and awareness of the systemic nature of climate action, but that opaque causal mechanisms and limited social contextualization impede deeper systems understanding and actionability. We position Co-QA as a formative design cycle that generates actionable implications for design. Specifically, we recommend (1) micro-explanations and visualizations to enhance transparency; (2) narratives and impact visuals to improve representational relatability; (3) broadened policy space to support personally relevant experimentation; and (4) implementation framing and pathways to bolster feasibility and actionability of climate solutions. We contend that Co-QA helps advance transparent, participatory, and evidence-based standards for evaluating and designing simulation-based ILEs for climate education.