Harnessing Artificial Intelligence for Scalable Evidence Synthesis in Reviews: Application in a Bibliometric Analysis of Physical Activity Technologies

Introduction: Artificial intelligence (AI) tools offer promising opportunities to support evidence synthesis at scale. This study presents a novel AI-human hybrid screening approach to a large-scale bibliometric analysis of technologies promoting physical activity.Methods: Records (n = 28,957) were retrieved from electronic databases and screened using ASReview, an open-source machine learning tool. Over 100 seed articles trained the model. Screening followed an established framework with four phases. Phase 1 involved random screening of 1% of records (n = 290) to inform stopping rules. In Phase 2, three reviewers screened 3,994 records with active learning, stopping when pre-established heuristics were met. Phase 3 applied a second model to unlabelled records (n = 410), identifying 53 additional studies. Phase 4 re-ranked excluded records, yielding 226 additional studies. Final screening yielded 3,183 records, of which 2,585 were retained for analysis.Results: Only 18% of records required manual screening, saving an estimated 592 hours. Iterative model training supported early prioritisation of relevant studies and well-defined stopping rules provided a justifiable endpoint for screening. A multi-model strategy, including re-screening excluded and unlabelled records, safeguarded against false exclusions.Conclusion: The AI-assisted approach was feasible, efficient, and scalable, and provides practical guidance for future large reviews.