Operational Excellence - AI as Third Peer Reviewer
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
This response to the Request for Information (RFI) Inviting Comments on the NIH Artificial Intelligence (AI) Strategy with the Notice Number: NOT-OD-25-117 is motivated by the prevailing poor research quality and equally low ethical standards in the life sciences.A conversation with Google’s Gemini language model in May and June 2025 on AI's potential applications in the biomedical sciences culminated in four essays, which are presented below. The first emphasizes that the lack of robust training data for large language/reasoning models directly impacts their output (GIGO, Garbage In, Garbage Out). The second attributes poor research conduct to human biases and recommends incorporating AI into the peer review process to address the reproducibility crisis (AI Red Teaming). The benefits of AI peer reviewers evaluating grant proposals and research articles alongside human peers are highlighted in the third essay. The final essay contemplates that AI will ultimately still reflect only human ideas. These thoughts highlight AI's potential to address challenges of poor research quality and low ethical conduct in the life sciences. This response specifically touches on points:4. Operational Excellence (Opportunities for AI to improve NIH “customer” experiences),6. Reproducibility & Trust (Community-driven approaches), and2. Research & Innovation Action (Mechanisms to enhance reproducibility) of the RFI.It is my sincere hope these contributions will support the NIH’s AI Strategy in pursuit of more rigorous and reproducible biomedical research and scientific excellence.