Exam Security and Technology Governance Questionnaire (ESTG-Q): A Modular Instrument with an Item-Level Attitude-Marking Mechanism for Exam Integrity Research
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background: High-stakes examinations are increasingly challenged by technology-enabled cheating and by governance dilemmas in implementing countermeasures. While prior research has examined student misconduct and proctoring technologies, the governance perspective of examination administrators—the frontline implementers of integrity policies—remains under-examined. Methods: We developed the Exam Security and Technology Governance Questionnaire (ESTG‑Q), a modular instrument comprising 280 questions. Development was guided by a provisional domain map (initially drafted as 16 topical categories and 22 hypothesized dimensions) that remains subject to refinement as the broader program progresses. For administration and reporting, items are presented in numerical order and organized into chapter-aligned thematic blocks; these blocks are navigation/reporting units rather than validated latent constructs. The instrument combines 233 Likert-type items with scenario-based dilemmas and contextual/background items. Item development followed domain mapping and iterative review. To improve traceability during item refinement, we used an LLM-assisted, CVI-style content-relevance screening (six independent runs under a fixed 4-point rubric) as additional evidence alongside human judgment. An optional “thumbs-up/thumbs-down/abstain” attitude-marking feature captured respondent-centered item acceptability. The instrument was fielded in a national sample of examination administrators in China (N = 207). Results: The evidence indicated high item relevance agreement (>90% of items had I-CVI ≥ 0.75; S-CVI/Ave = 0.92 for all items and 0.97 for core-construct items). Whole-instrument internal consistency across the 233 Likert-type items was high (Cronbach’s α = 0.963). A seven-item Cheating Attribution subscale (Q236–Q242) showed acceptable reliability (α = 0.723) and a two-factor structure in EFA (Factor 1 = institutional deterrence beliefs, 39.77%; Factor 2 = role-based moral permissiveness, 28.09%; cumulative = 67.86%), supported by CFA (χ²/df = 2.10, CFI = 0.97, RMSEA = 0.06). Conclusions: The ESTG-Q enables modular, profile-based assessment of exam security governance at scale while documenting item-level acceptability in sensitive integrity contexts. It supports diagnostic use beyond a single total score and facilitates comparison between deterrence-oriented beliefs and role-based discretionary judgments.