AutoMetaCoder: An AI Agent for Meta-analysis Coding

Coding in Meta-analysis is a critical component in cumulative scientific research. However, it is labor-intensive and susceptible to human error. With recent advances in large language models (LLMs), this preprint introduces AutoMetaCoder, an open-source AI agent designed to support meta-analytic coding through a structured, rule-governed, and human-in-the-loop workflow. AutoMetaCoder decomposes coding into modular stages, including study screening, multi-level information extraction, and standardized output organization for downstream analyses, while explicitly logging interactions between researcher inputs and model outputs to enhance transparency and auditability. We provide a detailed walkthrough of the system and present initial validation results based on published psychological meta-analyses. AutoMetaCoder achieves high retrieval and accuracy rates (97% overall) when extracting paper, sample, variable, and effect size related information across 55 primary studies with over 5,000 coding entries. These findings suggest that agentic AI workflows can substantially reduce manual coding burden while preserving methodological rigor and transparency in meta-analysis.