Bias in Standardized Testing? An Item Response Theory Analysis of Cultural and Religious Influences of a Multi-Denominational Educational Systems in Ghana and Botswana

Standardized assessments are widely used to measure student achievement; however, they often fail to account for cultural and religious influences that may affect item functioning. This study investigated the extent to which cultural and religious factors influenced test performance in Ghana and Botswana using Item Response Theory (IRT) and Differential Item Functioning (DIF) analysis. A mixed-methods approach was employed, combining quantitative DIF analysis of standardized test data from 1,200 students (600 from Ghana and 600 from Botswana) with qualitative insights from 30 student interviews and 6 focus group discussions. Findings revealed that 27% of reading comprehension items and 21% of social studies items exhibited significant DIF (p < 0.01), favoring students whose cultural and religious backgrounds aligned with the content of test items. Mathematics items displayed fewer instances of DIF (9%), but word problems that referenced religious or cultural practices led to performance disparities. In particular, test items referencing Christian parables had a large DIF effect (effect size > 0.64), disadvantaging Muslim and secular students, while questions on chieftaincy and traditional leadership exhibited moderate DIF (effect size 0.45–0.55), affecting urban students unfamiliar with these concepts. Qualitative data reinforced these findings, as students expressed that familiarity with religious and cultural references helped them engage with test items more effectively. Some Muslim students found Christian-based passages challenging, while urban students reported difficulty understanding traditional folklore-related questions. Teachers and assessment experts raised concerns that standardized tests often reflect dominant cultural narratives, potentially disadvantaging minority groups. The study highlights the need for more inclusive assessment practices that minimize cultural bias and ensure equitable educational opportunities for all students. Key recommendations include implementing DIF analysis in test development, using culturally neutral content, engaging diverse stakeholders in assessment design, adopting alternative assessment methods, and training educators on bias in testing. These findings contribute to the broader discourse on fairness in educational assessment and offer practical strategies for improving testing practices in multi-denominational societies such as Ghana and Botswana.