Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
To audit the transparent and open science standards of health and medical sciences journal policies and explore the impact of the COVID-19 pandemic.
Design
Repeat cross-sectional study.
Setting
19 journals listed in Google Scholar's Top Publications for health and medical sciences.
Participants
Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences, Science Translational Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet, The Lancet Oncology, and The New England Journal of Medicine.
Main outcome measures
We used the Transparency and Openness Promotion (TOP) guideline and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosing conflicts of interest (COIs) to evaluate journals standards.
Results
TOP scores slightly improved during the COVID-19 pandemic, from a median of 5 (IQR: 2–12.5) out of a possible 24 points in February 2020 to 7 (IQR: 4–12) in May 2021, but overall, scores were very low at both time points. Journal policies scored highest for their adherence to data transparency and scored lowest for preregistration of study protocols and analysis plans and the submission of replication studies. Most journals fulfilled all ICMJE provisions for reporting COIs before (84%; n = 16) and during (95%; n = 18) the COVID-19 pandemic.
Conclusions
The COVID-19 pandemic has highlighted the importance of practising open science. However, requirements for open science practices in audited policies were overall low, which may impede progress in health and medical research. As key stakeholders in disseminating research, journals should promote a research culture of greater transparency and more robust open science practices.
Article activity feed
-
-
SciScore for 10.1101/2022.01.26.22269868: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: To examine the journal’s requirements for authors to disclose conflict of interests (COIs), we used the four standards in the International Committee of Medical Journal Editors (ICMJE) disclosure form39. Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Table 2: Resources
Software and Algorithms Sentences Resources Sampling procedure: On 31 January 2020, we sampled 19 journals listed in Google Scholar’s Top Publications for the health and medical sciences subcategory35, which at this time were (in alphabetical order): Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of … SciScore for 10.1101/2022.01.26.22269868: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: To examine the journal’s requirements for authors to disclose conflict of interests (COIs), we used the four standards in the International Committee of Medical Journal Editors (ICMJE) disclosure form39. Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Table 2: Resources
Software and Algorithms Sentences Resources Sampling procedure: On 31 January 2020, we sampled 19 journals listed in Google Scholar’s Top Publications for the health and medical sciences subcategory35, which at this time were (in alphabetical order): Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences (PNAS), Science Translational Medicine, The British Medical Journal (BMJ), The Journal of the American Medical Association (JAMA), The Lancet, The Lancet Oncology, and The New England Journal of Medicine (NEJM). Google Scholar’ssuggested: NoneNeuronsuggested: (NEURON, RRID:SCR_005393)Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Strengths and limitations: While the TOP guideline is a valuable tool for assessing journals’ transparency and openness standards, there are some limitations. Firstly, the presence of imprecise and undefined terms when describing the required degree of adherence to a standard, such as ‘should’, ‘strongly encourage’, ‘recommend’ or ‘expect’, can lead to difficulties in allocating scores. To mitigate this, we conducted the scoring independently in dual and discussed scoring with a third study author to resolve differences in scoring. Our study assessed the policies of journals and not adherence of their published articles to these policies. The TOP guideline may not be applicable or flexible when examining the policies of specialized journals, such as the Cochrane Database of Systematic Reviews, which we excluded from our study. Nevertheless, the TOP guideline applies equally to journals that publish all study designs and provides a useful overview of how journals can improve the quality of research it publishes. Future research: Our methods can be replicated by others to assess the transparency and openness standards of journals in different research fields. This information can be provided to journal editors to encourage them to improve their policies. Future research should review the degree of transparency of research published in journals to assess compliance with journal policies. Furthermore, the impact of poor transparency and open science practices on individual and so...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-