The methodological quality is insufficient in clinical practice guidelines in the context of COVID-19: systematic review

  1. Tanja A. Stamm
  2. Margaret R. Andrews
  3. Erika Mosor
  4. Valentin Ritschl
  5. Linda C. Li
  6. Jasmin K. Ma
  7. Adalberto Campo‐Arias
  8. Sarah Baker
  9. Nicola W. Burton
  10. Mohammad Eghbali
  11. Natalia Fernandez
  12. Ricardo J.O. Ferreira
  13. Gabriele Gäbler
  14. Souzi Makri
  15. Sandra Mintz
  16. Rikke Helene Moe
  17. Elizabeth Morasso
  18. Susan L. Murphy
  19. Simiso Ntuli
  20. Maisa Omara
  21. Miguel Angel Simancas-Pallares
  22. Jen Horonieff
  23. Gerald Gartlehner

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

No abstract available

Article activity feed

  1. Version published to 10.1016/j.jclinepi.2021.03.005
  2. ScreenIT

    SciScore for 10.1101/2020.06.19.20134767: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Data Sources and Searches: We performed a rapid systematic review informed by the Cochrane Rapid Reviews Interim Guidance from the Cochrane Rapid Reviews Methods Group (6) in collaboration with Cochrane Austria, located at the Danube University Krems.
    Cochrane Rapid Reviews
    suggested: None
    Cochrane Austria
    suggested: None
    An international expert task force including representatives from each of the six WHO regions (Europe, America, Africa, Eastern-Mediterranean, South-East Asia, Western Pacific) defined research questions, keywords, Medical Subject Headings (MeSH) terms and a search strategy for the rapid systematic review.
    MeSH
    suggested: (MeSH, RRID:SCR_004750)
    We searched the MEDical Literature Analysis and Retrieval System OnLINE (MEDLINE) [PubMed], the Cumulative Index to Nursing & Allied Health Literature (CINAHL
    MEDLINE
    suggested: (MEDLINE, RRID:SCR_002185)
    It comprises 23 items assessing six domains including scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence.
    scope
    suggested: (Scope, RRID:SCR_017454)
    Data Synthesis and Analysis: Descriptive statistics were calculated for relevant extraction fields for the entire dataset using Microsoft Excel, including geographic affiliation of authors, publication status, type of recommendation, target population, focus of recommendation, disease/condition, and setting.
    Microsoft Excel
    suggested: (Microsoft Excel, RRID:SCR_016137)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The quality and methodological limitations of the guidelines should be placed in the context of the current unprecedented situation and the human resources required to produce high-quality guidelines in a timely manner. While systematically developed statements reflecting the current state of knowledge and supporting health professionals in their work during the COVID-19 pandemic have been urgently needed, basic methodological standards in the guideline development process are nevertheless essential to avoid misleading information and potentially harmful actions for patients and the healthcare system as a whole. One S3 guideline (19) with high overall methodological ratings had been published in calendar week six from when we found the first guidelines in our search (Figure 3), demonstrating that it was possible to produce high-quality work in a short time. Taking more time to develop guidelines therefore may not always lead to a more rigorous methodological basis. Rather, the methodological commitment, expertise of the authors and engagement of appropriate stakeholders might be important for a rigorous development process, independent of the time period after the outbreak when the guideline was published. Future guideline developers should thus be encouraged to consider rigorous methodological standards; further research could monitor the evolving methodological quality of the guidelines and their updates over time. Similar to our findings of clinical practice guidelines fal...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.06.19.20134767v1 on medRxiv