Derivation and Validation of Clinical Prediction Rules for COVID-19 Mortality in Ontario, Canada
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Background
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is currently causing a high-mortality global pandemic. The clinical spectrum of disease caused by this virus is broad, ranging from asymptomatic infection to organ failure and death. Risk stratification of individuals with coronavirus disease 2019 (COVID-19) is desirable for management, and prioritization for trial enrollment. We developed a prediction rule for COVID-19 mortality in a population-based cohort in Ontario, Canada.
Methods
Data from Ontario’s provincial iPHIS system were extracted for the period from January 23 to May 15, 2020. Logistic regression–based prediction rules and a rule derived using a Cox proportional hazards model were developed and validated using split-halves validation. Sensitivity analyses were performed, with varying approaches to missing data.
Results
Of 21 922 COVID-19 cases, 1734 with complete data were included in the derivation set; 1796 were included in the validation set. Age and comorbidities (notably diabetes, renal disease, and immune compromise) were strong predictors of mortality. Four point-based prediction rules were derived (base case, smoking excluded, long-term care excluded, and Cox model–based). All displayed excellent discrimination (area under the curve for all rules > 0.92) and calibration (P > .50 by Hosmer-Lemeshow test) in the derivation set. All performed well in the validation set and were robust to varying approaches to replacement of missing variables.
Conclusions
We used a public health case management data system to build and validate 4 accurate, well-calibrated, robust clinical prediction rules for COVID-19 mortality in Ontario, Canada. While these rules need external validation, they may be useful tools for management, risk stratification, and clinical trials.
Article activity feed
-
-
SciScore for 10.1101/2020.06.21.20136929: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement IRB: The study was approved by the research ethics board of the University of Toronto. Randomization Statistical analysis: We randomly assorted cases into derivation and validation sets. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Our analysis had many limitations; the use of a public health record system not …
SciScore for 10.1101/2020.06.21.20136929: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement IRB: The study was approved by the research ethics board of the University of Toronto. Randomization Statistical analysis: We randomly assorted cases into derivation and validation sets. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Our analysis had many limitations; the use of a public health record system not explicitly designed as a research tool means that we lack laboratory and radiological results that have been useful in other prediction models (23, 29). Furthermore, missing data was a significant limitation of our dataset, although our models appeared robust even with random replacement of predictors and outcomes that should bias associations towards the null. In that sense, the ability to derive simple, accurate and parsimonious rules, which perform well in split-halves validation, despite limitations in our dataset, may suggest generalizability of application outside Ontario. We hope that other groups will evaluate our rules in other settings. In summary, we developed and internally validated a prediction rule for COVID-19 mortality using a large and detailed public health line list in the Canadian province of Ontario. The rule was well calibrated and discriminated well and was robust in sensitivity analyses to assess the impact of missing information on predictor variables. If externally validated, this rule might facilitate decision making during future epidemic waves.
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
