LLM-Assisted Replication as Scientific Infrastructure
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Large language models (LLMs) are rapidly accelerating scientific production, from literature synthesis to automated analysis. Yet this expansion risks creating a verification gap, in which the volume of scientific claims outpaces the community’s capacity to check their reproducibility. We argue that the same LLM capabilities driving scientific output can be redirected toward scalable verification. As a demonstration, we share insights and lessons from our attempt to reproduce the core results of popular classical papers across disciplines, which yielded not only successful cases but also failures due to underspecified methodological details. This means that automated replication does not adjudicate scientific truth, but it localizes discrepancies and documentation gaps, lowering the cost of computational reproducibility checks. We thereby propose embedding LLM-assisted replication across the research lifecycle, from pre-submission quality check, journal-integrated verification, post-publication audits, to forensic reconstruction of legacy studies. To prevent misuse and preserve trust, we call for transparent standards and community governance. If institutionalized responsibly, AI can serve not only to generate science, but to scale its self-correction.