Validation Gaps in Psychological and Social Science Instruments: A Systematic Review and Meta-Analysis (2020–2025)

Background: Quantitative instruments are essential in psychology and social sciences, yet persistent validation deficiencies may compromise research integrity and evidence-based practice.Objective: To systematically quantify validation compliance rates across psychological and social science research, identifying systematic deficiencies in measurement practices.Methods: Following PRISMA 2020 and COSMIN 2024 guidelines, the author searched four major databases for empirical validation studies (January 1, 2020–May 31, 2025). Random-effects meta-analyses employed restricted maximum likelihood estimation with a logit transformation; three-level models were used to address dependencies. Heterogeneity was assessed via I², τ², and 95% prediction intervals; publication bias was examined using PET-PEESE regression and selection models.Results: 248 studies (34,567 validation procedures; 52 countries) were included. Measurement invariance compliance declined by level: configural 68.4% [48.1–84.7], metric 41.7% [22.8–63.4], scalar 23.8% [11.2–44.7], strict 12.3% [5.1–27.4]. Sample size adequacy was achieved in 47.2% [24.1–71.8]. Open science practices showed low adoption rates: preregistration (19.7% [9.1–37.4%]), open data (31.4% [14.2–55.8%]), and code availability (15.8% [7.2–31.1%]). Geographic and disciplinary disparities emerged, with cross-domain instruments exhibiting lower validation quality. Publication bias adjustment reduced scalar invariance compliance to 18.3%.Conclusions: Validation deficiencies are prevalent, posing a significant threat to the integrity of research. Fewer than 25% verify scalar invariance, under 50% meet sample size criteria, and transparency practices remain underutilized. Coordinated interventions, including training, editorial policies, and funding support, are crucial for strengthening validation standards.