Development and external validation of prognostic models for COVID-19 to support risk stratification in secondary care
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Existing UK prognostic models for patients admitted to the hospital with COVID-19 are limited by reliance on comorbidities, which are under-recorded in secondary care, and lack of imaging data among the candidate predictors. Our aims were to develop and externally validate novel prognostic models for adverse outcomes (death and intensive therapy unit (ITU) admission) in UK secondary care and externally validate the existing 4C score.
Design
Candidate predictors included demographic variables, symptoms, physiological measures, imaging and laboratory tests. Final models used logistic regression with stepwise selection.
Setting
Model development was performed in data from University Hospitals Birmingham (UHB). External validation was performed in the CovidCollab dataset.
Participants
Patients with COVID-19 admitted to UHB January–August 2020 were included.
Main outcome measures
Death and ITU admission within 28 days of admission.
Results
1040 patients with COVID-19 were included in the derivation cohort; 288 (28%) died and 183 (18%) were admitted to ITU within 28 days of admission. Area under the receiver operating characteristic curve (AUROC) for mortality was 0.791 (95% CI 0.761 to 0.822) in UHB and 0.767 (95% CI 0.754 to 0.780) in CovidCollab; AUROC for ITU admission was 0.906 (95% CI 0.883 to 0.929) in UHB and 0.811 (95% CI 0.795 to 0.828) in CovidCollab. Models showed good calibration. Addition of comorbidities to candidate predictors did not improve model performance. AUROC for the International Severe Acute Respiratory and Emerging Infection Consortium 4C score in the UHB dataset was 0.753 (95% CI 0.720 to 0.785).
Conclusions
The novel prognostic models showed good discrimination and calibration in derivation and external validation datasets, and performed at least as well as the existing 4C score using only routinely collected patient information. The models can be integrated into electronic medical records systems to calculate each individual patient’s probability of death or ITU admission at the time of hospital admission. Implementation of the models and clinical utility should be evaluated.
Article activity feed
-
-
SciScore for 10.1101/2021.01.25.21249942: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Strengths and limitations: The UHB dataset represents one of the largest and most ethnically diverse patient cohorts within the UK. Additionally, as part of the early UHB response to the COVID-19 pandemic, the hospital trust ensured that, upon admission, all patients underwent a wide range of investigations to support international …
SciScore for 10.1101/2021.01.25.21249942: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Strengths and limitations: The UHB dataset represents one of the largest and most ethnically diverse patient cohorts within the UK. Additionally, as part of the early UHB response to the COVID-19 pandemic, the hospital trust ensured that, upon admission, all patients underwent a wide range of investigations to support international research efforts examining prognostic markers. This allowed us to examine a wide range of possible predictors (63 candidate predictors after exclusions). Lastly, a strength of this study was the good performance, in terms of both discrimination and calibration, of the simplified, reduced model in an externally validated cohort (CovidCollab), indicating its suitability for wider use, including potentially in LMICs. Despite the strengths, the findings must be considered in light of the study’s limitations. Although we were able to use a derivation dataset from UHB with low levels of missing data, the overall sample size was relatively small compared to that of the ISARIC study and was limited to one UK geographical location. However, we were able to externally validate the model in a larger external cohort. A second limitation was that in the external validation cohort we were unable to examine all of the predictors included in the original full UHB model due to only a reduced set of candidate predictors being available in CovidCollab. Nevertheless, the model performed well and the results suggest it may be applicable in a wide range of datasets wher...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
