SOFA score performs worse than age for predicting mortality in patients with COVID-19

  1. Raphael A. G. Sherak
  2. Hoomaan Sajjadi
  3. Naveed Khimani
  4. Benjamin Tolchin
  5. Karen Jubanyik
  6. R. Andrew Taylor
  7. Wade Schulz
  8. Bobak J. Mortazavi
  9. Adrian D Haimovich

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The use of the Sequential Organ Failure Assessment (SOFA) score, originally developed to describe disease morbidity, is commonly used to predict in-hospital mortality. During the COVID-19 pandemic, many protocols for crisis standards of care used the SOFA score to select patients to be deprioritized due to a low likelihood of survival. A prior study found that age outperformed the SOFA score for mortality prediction in patients with COVID-19, but was limited to a small cohort of intensive care unit (ICU) patients and did not address whether their findings were unique to patients with COVID-19. Moreover, it is not known how well these measures perform across races.

In this retrospective study, we compare the performance of age and SOFA scores in predicting in-hospital mortality across two cohorts: a cohort of 2,648 consecutive adult patients diagnosed with COVID-19 who were admitted to a large academic health system in the northeastern United States over a 4-month period in 2020 and a cohort of 75,601 patients admitted to one of 335 ICUs in the eICU database between 2014 and 2015.

Among the COVID-19 cohort, age (area under receiver-operating characteristic curve (AU-ROC) 0.795, 95% CI 0.762, 0.828) had a significantly better discrimination than SOFA score (AU-ROC 0.679, 95% CI 0.638, 0.721) for mortality prediction. Conversely, age (AU-ROC 0.628 95% CI 0.608, 0.628) underperformed compared to SOFA score (AU-ROC 0.735, 95% CI 0.726, 0.745) in non-COVID-19 ICU patients in the eICU database. There was no difference between Black and White COVID-19 patients in performance of either age or SOFA Score. Our findings bring into question the utility of SOFA score-based resource allocation in COVID-19 crisis standards of care.

Author Summary

The COVID-19 pandemic has prompted hospitals to develop protocols for allocating resources if the number of patients exceed their capacity in order to save as many lives as possible. Many of these protocols use the Sequential Organ Failure Assessment (SOFA) score to identify patients who are unlikely to survive and thus should be deprioritized for care. There are concerns that the SOFA score may not accurately predict mortality in patients with COVID-19 or perform better in one racial group over another. We asked whether a simple measure, patient age, could better predict mortality than SOFA score in a group of adult patients admitted to a large academic health system in 2020. To see if any findings are unique to patients with COVID-19, we performed the same analysis in a group of adult patients taken from the eICU database, a large publicly available dataset that was collected prior to the COVID-19 pandemic. We found that age was better than SOFA score at predicting patient mortality in patients with COVID-19, but not in patients without COVID. For COVID-19, neither age or SOFA score performed better in one racial group over another. Caution is needed when applying an established disease severity index model to a new illness.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2022.05.02.22274575: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: This study was exempted by the Yale University Institutional Review Board (2000027747).
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    The SOFA scores on eICU were calculated and extracted using Python’s Pandas library, based on its standard definition on the values of creatinine, bilirubin, platelets, fraction of inspired oxygen, partial pressure of inspired oxygen, Glasgow Coma Scale, mean arterial pressure, and mechanical ventilation status.
    Python’s
    suggested: (PyMVPA, RRID:SCR_006099)
    All analyses were performed in Python (Version 3.7.7) and R (Version 1.4.1717).
    Python
    suggested: (IPython, RRID:SCR_001658)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study has several limitations. The cohort of COVID-19 patients was restricted to a single health system in the northeast US which may not have a comparable population to COVID-19 patients at other academic health systems or those in the eICU database. Due to sample size, we also restricted our analysis to two racial groups and did not consider patients that identified as multi-racial. We did not consider sex or its potential interaction with race or age in our study. Furthermore, an unknown proportion of patients in the COVID-19 cohort were too critically ill to answer demographic questions and had their race and sex recorded by a hospital clerk based on assumption. Additionally, this study only describes the performance of max SOFA score within 24 hours of hospital admission. Many COVID-19 positive patients present to the hospital with respiratory complaints and develop multisystem organ dysfunction later in their disease course (29). However, information on SOFA subscores in the cohort of COVID-19 patients were not available so we were unable to test this hypothesis. SOFA score may have greater utility for predicting mortality with serial measurements or later in disease course (2). As the COVID-19 pandemic continues well into 2022, SOFA score continues to feature prominently in guidelines for crisis standards of care (30). This study suggests caution should be used when considering SOFA score as a prognostic tool, as it has limited prognostic performance.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2022.05.02.22274575v1 on medRxiv