MAPEVAScR: a four-phase framework integrating transparent rule based text mining, evidence gap mapping and a mandatory human validation gate for scoping reviews framework development and empirical validation in occupational health.
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background. Automation is increasingly used to reduce the screening workload of evidence syntheses, but its error is real, variable and rarely reported. The 2025 joint position statement of Cochrane, Campbell, JBI and the Collaboration for Environmental Evidence mandates human oversight of artificial intelligence in evidence synthesis, yet, as of today, no international guideline requires quantitative measurement and reporting of automated-screening error against a human gold standard. Methods. We developed MAPEVAScR (MAPeo de EVidencia con VAlidacion humana), a four phase framework for scoping reviews: (A) datalake construction with deterministic, traceable deduplication; (B) transparent, record level auditable rule based text mining with confidence tiers; (C) evidence gap mapping that feeds back human screening prioritisation; and (D) a mandatory human validation gate verifying 100% of the candidate corpus and issuing a quantitative screening error report (per tier precision, confusion matrix, falsepositive families, documented changes to conclusions). The framework was empirically validated on a scoping review of epilepsy and occupational fitness (2015 to 2025; PubMed, Scopus, Web of Science): a funnel of 2,008 → 1,435 → 1,114 → 230 records was reduced to 127 human confirmed inclusions (present study). Results. The rule based filter achieved a global precision of 55.2% (Tier A 75.5%, Tier B 37.5%, Tier C 45.8%); human validation identified 103 false positives classifiable into seven reproducible error families, and 107 of 127 included records required human correction of automated thematic labels. Without the gate, the review's central thematic conclusion would have been published reversed: automated labelling ranked Aptitude/employability third (12.6% of records), whereas after validation it ranked first (73.2%) (present study). Conclusions. A mandatory human validation gate with quantitative error metrics closes a reporting gap that current guidelines leave open: in the present study it prevented publication of an inverted synthesis. We propose the screening error report as a minimum reporting standard for any evidence synthesis using automated screening. Keywords: scoping review; text mining; human validation; automation bias; evidence gap map; occupational health; epilepsy; PRISMAScR