Development and validation of multivariable prediction models for adverse COVID-19 outcomes in patients with IBD

  1. John Sperger
  2. Kushal S Shah
  3. Minxin Lu
  4. Xian Zhang
  5. Ryan C Ungaro
  6. Erica J Brenner
  7. Manasi Agrawal
  8. Jean-Frédéric Colombel
  9. Michael D Kappelman
  10. Michael R Kosorok

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Develop an individualised prognostic risk prediction tool for predicting the probability of adverse COVID-19 outcomes in patients with inflammatory bowel disease (IBD).

Design and setting

This study developed and validated prognostic penalised logistic regression models using reports to the international Surveillance Epidemiology of Coronavirus Under Research Exclusion for Inflammatory Bowel Disease voluntary registry from March to October 2020. Model development was done using a training data set (85% of cases reported 13 March–15 September 2020), and model validation was conducted using a test data set (the remaining 15% of cases plus all cases reported 16 September–20 October 2020).

Participants

We included 2709 cases from 59 countries (mean age 41.2 years (SD 18), 50.2% male). All submitted cases after removing duplicates were included.

Primary and secondary outcome measures

COVID-19 related: (1) Hospitalisation+: composite outcome of hospitalisation, ICU admission, mechanical ventilation or death; (2) Intensive Care Unit+ (ICU+): composite outcome of ICU admission, mechanical ventilation or death; (3) Death. We assessed the resulting models’ discrimination using the area under the curve of the receiver operator characteristic curves and reported the corresponding 95% CIs.

Results

Of the submitted cases, a total of 633 (24%) were hospitalised, 137 (5%) were admitted to the ICU or intubated and 69 (3%) died. 2009 patients comprised the training set and 700 the test set. The models demonstrated excellent discrimination, with a test set area under the curve (95% CI) of 0.79 (0.75 to 0.83) for Hospitalisation+, 0.88 (0.82 to 0.95) for ICU+ and 0.94 (0.89 to 0.99) for Death. Age, comorbidities, corticosteroid use and male gender were associated with a higher risk of death, while the use of biological therapies was associated with a lower risk.

Conclusions

Prognostic models can effectively predict who is at higher risk for COVID-19-related adverse outcomes in a population of patients with IBD. A free online risk calculator ( https://covidibd.org/covid-19-risk-calculator/ ) is available for healthcare providers to facilitate discussion of risks due to COVID-19 with patients with IBD.

Article activity feed

  1. Version published to 10.1136/bmjopen-2021-049740
  2. ScreenIT

    SciScore for 10.1101/2021.01.15.21249889: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    BlindingReporters were not explicitly informed of what data could be used as predictors or outcomes, but being a voluntary registry, reporters were not blinded.
    Power Analysisnot detected.
    Sex as a biological variableAs only one patient had reported a gender other than male or female, only two genders were considered in the analysis.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: Thank you for sharing your code.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitations: The data for this study comes from a voluntary registry, and the sampling process is unknown. Reported cases may underrepresent both low-risk asymptomatic cases and severely ill patients who may be hospitalized at an outside hospital or die without their healthcare provider’s knowledge. Our results are associational, not causal — when using the online risk calculator, healthcare providers should not use it to answer “what-if” questions (e.g., how an individual’s risk would change if they altered the medications they were taking) which are inherently causal questions33. While the registry has a wealth of clinical data, it does not collect granular data on many social determinants of health. Additionally, insurance status is not collected, which, for patients in the US, likely factors into the decision making of whether to visit a hospital. Lastly, we cannot compare the risk of adverse COVID-19 outcomes in IBD patients to that in the general population.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.01.15.21249889v1 on medRxiv