Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study

This article has been Reviewed by the following groups

Read the full article See related articles

Abstract

To audit the transparent and open science standards of health and medical sciences journal policies and explore the impact of the COVID-19 pandemic.

Design

Repeat cross-sectional study.

Setting

19 journals listed in Google Scholar's Top Publications for health and medical sciences.

Participants

Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences, Science Translational Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet, The Lancet Oncology, and The New England Journal of Medicine.

Main outcome measures

We used the Transparency and Openness Promotion (TOP) guideline and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosing conflicts of interest (COIs) to evaluate journals standards.

Results

TOP scores slightly improved during the COVID-19 pandemic, from a median of 5 (IQR: 2–12.5) out of a possible 24 points in February 2020 to 7 (IQR: 4–12) in May 2021, but overall, scores were very low at both time points. Journal policies scored highest for their adherence to data transparency and scored lowest for preregistration of study protocols and analysis plans and the submission of replication studies. Most journals fulfilled all ICMJE provisions for reporting COIs before (84%; n  =  16) and during (95%; n  =  18) the COVID-19 pandemic.

Conclusions

The COVID-19 pandemic has highlighted the importance of practising open science. However, requirements for open science practices in audited policies were overall low, which may impede progress in health and medical research. As key stakeholders in disseminating research, journals should promote a research culture of greater transparency and more robust open science practices.

Article activity feed

  1. SciScore for 10.1101/2022.01.26.22269868: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: To examine the journal’s requirements for authors to disclose conflict of interests (COIs), we used the four standards in the International Committee of Medical Journal Editors (ICMJE) disclosure form39.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Sampling procedure: On 31 January 2020, we sampled 19 journals listed in Google Scholar’s Top Publications for the health and medical sciences subcategory35, which at this time were (in alphabetical order): Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences (PNAS), Science Translational Medicine, The British Medical Journal (BMJ), The Journal of the American Medical Association (JAMA), The Lancet, The Lancet Oncology, and The New England Journal of Medicine (NEJM).
    Google Scholar’s
    suggested: None
    Neuron
    suggested: (NEURON, RRID:SCR_005393)

    Results from OddPub: Thank you for sharing your code and data.


    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Strengths and limitations: While the TOP guideline is a valuable tool for assessing journals’ transparency and openness standards, there are some limitations. Firstly, the presence of imprecise and undefined terms when describing the required degree of adherence to a standard, such as ‘should’, ‘strongly encourage’, ‘recommend’ or ‘expect’, can lead to difficulties in allocating scores. To mitigate this, we conducted the scoring independently in dual and discussed scoring with a third study author to resolve differences in scoring. Our study assessed the policies of journals and not adherence of their published articles to these policies. The TOP guideline may not be applicable or flexible when examining the policies of specialized journals, such as the Cochrane Database of Systematic Reviews, which we excluded from our study. Nevertheless, the TOP guideline applies equally to journals that publish all study designs and provides a useful overview of how journals can improve the quality of research it publishes. Future research: Our methods can be replicated by others to assess the transparency and openness standards of journals in different research fields. This information can be provided to journal editors to encourage them to improve their policies. Future research should review the degree of transparency of research published in journals to assess compliance with journal policies. Furthermore, the impact of poor transparency and open science practices on individual and so...

    Results from TrialIdentifier: No clinical trial numbers were referenced.


    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.


    Results from JetFighter: We did not find any issues relating to colormaps.


    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.


    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.