Comparing expert assessments of research quality between the Global North and East Africa

This study investigates how expert assessments of the quality of behavioral science research vary across geographic and epistemic contexts. Using a form of structured peer review called the IDEA protocol, we use data from the repliCATS project (Global North cohort) and a follow-up study conducted by Busara (East Africa cohort), to compare how experts judge the quality, and especially the generalizability, of 80 behavioral science papers. Due to their greater familiarity with how other contexts might differ from Global North contexts, we expected that East African experts would express greater skepticism about the generalizability of these findings. Contrary to expectations, East African experts (n = 318) rated the papers as more credible on most metrics, including generalizability, than did the Global North experts (n = 384). We explore three interpretations of these findings: East African assessors possess unique contextual insights; Global North assessors apply stricter skepticism rooted in a broader crisis of confidence in behavioral science; or the comparison itself is invalid due to methodological issues or contextual mismatch. Our discussion illustrates that anyone interested in the cultural evolution of research practices needs to take into account differences and hierarchies between the research cultures of the Global North and the Global South.