Reproducibility and reporting practices in COVID-19 preprint manuscripts
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
The novel coronavirus, COVID-19, has sparked an outflow of scientific research seeking to understand the virus, its spread, and best practices in prevention and treatment. If this international research effort is going to be as swift and effective as possible, it will need to rely on a principle of open science. When researchers share data, code, and software and generally make their work as transparent as possible, it allows other researchers to verify and expand upon their work. Furthermore, it allows public officials to make informed decisions. In this study, we analyzed 535 preprint articles related to COVID-19 for eight transparency criteria and recorded study location and funding information. We found that individual researchers have lined up to help during this crisis, quickly tackling important public health questions, often without funding or support from outside organizations. However, most authors could improve their data sharing and scientific reporting practices. The contrast between researchers’ commitment to doing important research and their reporting practices reveals underlying weaknesses in the research community’s reporting habits, but not necessarily their science.
Article activity feed
-
SciScore for 10.1101/2020.03.24.20042796: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources On March 16, 2020, we harvested all preprint articles from the medRxivand bioRxiv databases related to COVID-19, for a total of 535 manuscripts. bioRxivsuggested: (bioRxiv, RRID:SCR_003933)Results from OddPub: Thank you for sharing your data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The COVID-19 pandemic has laid bare the medical systems of affected nations, exposing their strengths and weaknesses. Similarly, it …
SciScore for 10.1101/2020.03.24.20042796: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources On March 16, 2020, we harvested all preprint articles from the medRxivand bioRxiv databases related to COVID-19, for a total of 535 manuscripts. bioRxivsuggested: (bioRxiv, RRID:SCR_003933)Results from OddPub: Thank you for sharing your data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The COVID-19 pandemic has laid bare the medical systems of affected nations, exposing their strengths and weaknesses. Similarly, it has revealed patterns in the scientific community. In both the medical and research communities, COVID-19 has revealed the devoted, quality work of individuals but the many challenges of the larger systems that those individuals operate within. Since the outbreak of COVID-19 in 2019, researchers around the world have started exploring important health questions related to the virus, often without funding or outside support. Our research found that 8% of preprint articles received no funding. Furthermore, several new preprint articles related to COVID-19 are uploaded to medRxiv and bioRxiv each day. These actions reveal the dedication of scientists to provide their scientific information to decision-makers as quickly as possible. Yet, we found that most authors are not employing best reporting practices, despite the importance of open science during a health crisis. Many authors did not make their data readily accessible, did not share code, and did not engage in other important transparency practices. At a time when many scientists are going above and beyond to promote their research, these findings suggest that the scientific community may not recognize the role that certain criteria play in reproducibility and may not appreciate the extent to which open science promotes innovation and advancement. It may also signal that it is simply difficult ...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-