Development and external validation of a diagnostic multivariable prediction model for a prompt identification of cases at high risk for SARS-COV-2 infection among patients admitted to the emergency department

  1. Nicola Ughi
  2. Antonella Adinolfi
  3. Michel Chevallard
  4. Laura Belloli
  5. Michele Senatore
  6. Alessandro Toscano
  7. Andrea Bellone
  8. Cristina Giannattasio
  9. Paolo Tarsia
  10. Massimo Puoti
  11. Francesco Scaglione
  12. Fabrizio Colombo
  13. Michaela Bertuzzi
  14. Giuseppe Bettoni
  15. Davide Ferrazzi
  16. Alessandro Maloberti
  17. Armanda Dicuonzo
  18. Francesca Del Gaudio
  19. Claudio Rossetti
  20. Oscar Massimiliano Epis

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Background

An urgent need exists for an early detection of cases with a high-risk of SARS-CoV-2 infection, particularly in high-flow and -risk settings, such as emergency departments (EDs). The aim of this work is to develop and validate a predictive model for the evaluation of SARS-CoV-2 infection risk, with the rationale of using this tool to manage ED patients.

Methods

A retrospective study was performed by cross-sectionally reviewing the electronical case records of patients admitted to Niguarda Hospital or referred to its ED in the period 15 March to 24 April 2020.

Derivation sample was composed of non-random inpatients hospitalized on 24 April and admitted before 22 April 2020. Validation sample was composed of consecutive patients who visited the ED between 15 and 25 March 2020. The association between the dichotomic outcome and each predictor was explored by univariate analysis with logistic regression models.

Results

A total of 113 patients in the derivation sample and 419 in the validation sample were analyzed. History of fever, elder age and low oxygen saturation showed to be significant predictors of SARS-CoV-2 infection. The neutrophil count improves the discriminative ability of the model, even if its calibration and usefulness in terms of diagnosis is unclear.

Conclusion

The discriminatory ability of the identified models makes the overall performance suboptimal; their implementation to calculate the individual risk of infection should not be used without additional investigations. However, they could be useful to evaluate the spatial allocation of patients while awaiting the result of the nasopharyngeal swab.

Key Messages box

What is already known on this topic

1 year after the onset of the coronavirus disease 2019 (COVID-19) pandemic, the trend of its spread has not shown a substantial global reduction. An urgent need exists for efficient early detection of cases with a high risk of SARS-CoV-2 infection and a number of diagnostic prediction models have been developed, but a few models were externally validated in high-flow and –risk settings, such as emergency departments (EDs).

What this study adds

This study develops and validate predictive models for the evaluation of SARS-CoV-2 infection risk, with the rationale of using these tools to promptly manage patients who are afferent to the ED, allocating them accordingly to the risk of infection while awaiting swab result. History of fever, older age and low oxygen saturation showed to be significant predictors of the presence of SARS-CoV-2 infection. The use of laboratory findings, such as neutrophil count, showed to improve the discriminative ability of the model, even if its calibration and usefulness in terms of diagnosis is unclear.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2021.05.26.21257834: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: The study is conformed to Helsinki’s Declaration and was approved by the ethics committee Milano Area 3 (register number 249-13052020).
    Consent: An informed consent was provided by the enrolled participants.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power AnalysisThe estimated size of the validation sample for a one-sample two-sided mean test was 351 to achieve 80% power if the mean difference between C-statistics 0.03 with standard deviation 0.20, and the type I error 0.05 were assumed.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    All the analyses were performed using Stata Statistical Software Release 15 (StataCorp. 2017, College Station, TX: StataCorp LLC) and R (R Core Team 2018, R Foundation for Statistical Computing, Vienna, Austria).
    StataCorp
    suggested: (Stata, RRID:SCR_012763)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This study has several limitations. First, it was developed by using a case-control design and case analysis and data collection were retrospective and unblinded. However, since hard outcome and predictors like the detection of SARS-CoV-2 genome in the nasopharyngeal swab and age were used, a limited impact of bias on predictions may be assumed. Secondly, predictors like communal living and history of contact with a person diagnosed of SARS-CoV-2 infection were unmeasured, even though their relevance for diagnostic predictions has been still to be determined. Finally, the discriminatory ability of the models makes the overall performance still suboptimal and its implementation to calculate the individual risk of infection should not be used without additional investigations. However, it could be considered to evaluate the spatial allocation of the patients while awaiting the result of the nasopharyngeal swab, which is still the current reference standard for the diagnosis of SARS-Cov-2 infection. In conclusion, the development and validation of these models showed that prediction tools based on clinical findings like history of fever, age, and oxygen saturation at presentation may be accurate and robust to identify patients at high risk for a diagnosis of SARS-CoV-2 infection. Future external validation studies should be considered to evaluate if such models will be also robust to variable outcome prevalence, particularly in low proportions of infection, and to search for add...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2021.05.26.21257834v1 on medRxiv