Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

Abstract

Objective

To develop and validate a pragmatic risk score to predict mortality in patients admitted to hospital with coronavirus disease 2019 (covid-19).

Design

Prospective observational cohort study.

Setting

International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) World Health Organization (WHO) Clinical Characterisation Protocol UK (CCP-UK) study (performed by the ISARIC Coronavirus Clinical Characterisation Consortium—ISARIC-4C) in 260 hospitals across England, Scotland, and Wales. Model training was performed on a cohort of patients recruited between 6 February and 20 May 2020, with validation conducted on a second cohort of patients recruited after model development between 21 May and 29 June 2020 .

Participants

Adults (age ≥18 years) admitted to hospital with covid-19 at least four weeks before final data extraction.

Main outcome measure

In-hospital mortality.

Results

35 463 patients were included in the derivation dataset (mortality rate 32.2%) and 22 361 in the validation dataset (mortality rate 30.1%). The final 4C Mortality Score included eight variables readily available at initial hospital assessment: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, level of consciousness, urea level, and C reactive protein (score range 0-21 points). The 4C Score showed high discrimination for mortality (derivation cohort: area under the receiver operating characteristic curve 0.79, 95% confidence interval 0.78 to 0.79; validation cohort: 0.77, 0.76 to 0.77) with excellent calibration (validation: calibration-in-the-large=0, slope=1.0). Patients with a score of at least 15 (n=4158, 19%) had a 62% mortality (positive predictive value 62%) compared with 1% mortality for those with a score of 3 or less (n=1650, 7%; negative predictive value 99%). Discriminatory performance was higher than 15 pre-existing risk stratification scores (area under the receiver operating characteristic curve range 0.61-0.76), with scores developed in other covid-19 cohorts often performing poorly (range 0.63-0.73).

Conclusions

An easy-to-use risk stratification score has been developed and validated based on commonly available parameters at hospital presentation. The 4C Mortality Score outperformed existing scores, showed utility to directly inform clinical decision making, and can be used to stratify patients admitted to hospital with covid-19 into different management groups. The score should be further validated to determine its applicability in other populations.

Study registration

ISRCTN66726260

SciScore for 10.1101/2020.07.30.20165464: (What is this?)

Please note, not all rigor criteria are appropriate for all manuscripts.

Table 1: Rigor

Institutional Review Board Statement	not detected.
Randomization	not detected.
Blinding	not detected.
Power Analysis	not detected.
Sex as a biological variable	not detected.

Table 2: Resources

No key resources detected.

Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:

Strengths and limitations of this study: The ISARIC WHO CCP-UK study represents the largest prospectively collected covid-19 hospitalised patient cohort in the world and reflects the clinical data available in most economically developed healthcare settings. We developed a clinically applicable score with clear methodology and tested it against existing risk stratification scores in a large hospitalised patient cohort. It compared favourably to other prognostic tools, with good to excellent discrimination, calibration and performance characteristics. The 4C Mortality Score has several methodological advantages over current covid-19 prognostic scores. The use of penalised regression methods, an event-to-variable ratio greater than 100 reducing the risk of model over-fitting,44,45 and the use of clinical parameters at first assessment increases the clinical applicability of the score and limits use of highly selective predictors prevalent in other risk stratification scores.4,46 In addition, the sensitivity analyses demonstrated that score performance was robust. Our study has limitations. First, we were unable to evaluate the predictive performance of a number of existing scores that comprise a large number of parameters (for example APACHE II47), as well as several other covid-19 prognostic scores that include computed tomography findings or uncommonly measured biomarkers.5 In addition, several potentially relevant comorbidities, such as hypertension, previous myocardial infa...

Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

Identifier	Status	Title
ISRCTN66726260	NA	NA

Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

Results from JetFighter: We did not find any issues relating to colormaps.

Results from rtransparent:

Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
No protocol registration statement was detected.

Read the original source