External validation of risk scores to predict in-hospital mortality in patients hospitalized due to coronavirus disease 2019

  1. Shermarke Hassan
  2. Chava L. Ramspek
  3. Barbara Ferrari
  4. Merel van Diepen
  5. Raffaella Rossio
  6. Rachel Knevel
  7. Vincenzo la Mura
  8. Andrea Artoni
  9. Ida Martinelli
  10. Alessandra Bandera
  11. Alessandro Nobili
  12. Andrea Gori
  13. Francesco Blasi
  14. Ciro Canetta
  15. Nicola Montano
  16. Frits R. Rosendaal
  17. Flora Peyvandi

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

No abstract available

Article activity feed

  1. Version published to 10.1016/j.ejim.2022.06.005
  2. ScreenIT

    SciScore for 10.1101/2022.03.11.22271912: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: This study was approved by the Medical Ethics Committee of the Fondazione IRCCS Ca’ Granda Ospedale Maggiore Policlinico and the Institutional Review Board of the LUMC for observational studies.
    Sex as a biological variableThe 4C mortality score includes the following eight predictors, collected on the day of admission: age in years (categorical variable: <50, 50-59, 60-69, 70-79, ≥80); sex at birth (dichotomous variable: male, female); respiratory rate in breaths/min (categorical variable: >20, 20-29, ≥30); oxygen saturation on room air (dichotomous variable: ≥92%, <92%); Glasgow coma scale (dichotomous variable: 15 points, <15 points); urea (categorical variable: <7 mmol/L, ≥7 to ≤14 mmol/L,
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    (as this was not possible) Limitations: It has been suggested that the minimum sample size for external validation should be at least 100 events and 100 non-events. [35] The Leiden cohort had 41 deaths, falling below the number suggested by this rule of thumb. The sample size of the Lombardy cohort was acceptable for external validation. However, the Lombardy cohort consisted of patients that were enrolled in the first months of 2020. After this period, the incidence COVID-19 related mortality has changed a lot due to many treatment changes. This limits the applicability of the recalibrated 4C risk score as the population in which the scores were recalibrated may not be representative of the current patient population in 2022. Furthermore, some patients who were already very ill before being hospitalized for COVID-19 would have chosen to receive end-of-life care at home. Despite not dying in the hospital (in-hospital mortality was the study outcome), the 4C mortality score would have assigned these patients a very high predicted in-hospital mortality risk. This will have reduced the model performance (especially model discrimination), depending on the proportion of patients that received end-of-life care at home. Lastly, missing data may have influenced model performance. For example, the oxygen saturation on room air (a predictor in the 4C mortality score) was missing in more than half of all patients in both the Lombardy cohort and the Leiden cohort. This is most likely bec...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2022.03.11.22271912v1 on medRxiv