Protocol for the development and evaluation of a tool for predicting risk of short-term adverse outcomes due to COVID-19 in the general UK population

  1. Julia Hippisley-Cox
  2. Ash K. Clift
  3. Carol Coupland
  4. Ruth Keogh
  5. Karla Diaz-Ordaz
  6. Elizabeth Williamson
  7. Ewen M. Harrison
  8. Andrew Hayward
  9. Harry Hemingway
  10. Peter Horby
  11. Nisha Mehta
  12. Jonathan Benger
  13. Kamlesh Khunti
  14. David Speigelhalter
  15. Aziz Sheikh
  16. Jonathan Valabhji
  17. Ronan A. Lyons
  18. John Robson
  19. Calum Semple
  20. Frank Kee
  21. Peter Johnson
  22. Susan Jebb
  23. Tony Williams
  24. David Coggon

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Introduction

Novel coronavirus 2019 (COVID-19) has propagated a global pandemic with significant health, economic and social costs. Emerging emergence has suggested that several factors may be associated with increased risk from severe outcomes or death from COVID-19. Clinical risk prediction tools have significant potential to generate individualised assessment of risk and may be useful for population stratification and other use cases.

Methods and analysis

We will use a prospective open cohort study of routinely collected data from 1205 general practices in England in the QResearch database. The primary outcome is COVID-19 mortality (in or out-of-hospital) defined as confirmed or suspected COVID-19 mentioned on the death certificate, or death occurring in a person with SARS-CoV-2 infection between 24 th January and 30 th April 2020. Our primary outcome in adults is COVID-19 mortality (including out of hospital and in hospital deaths). We will also examine COVID-19 hospitalisation in children. Time-to-event models will be developed in the training data to derive separate risk equations in adults (19-100 years) for males and females for evaluation of risk of each outcome within the 3-month follow-up period (24 th January to 30 th April 2020), accounting for competing risks. Predictors considered will include age, sex, ethnicity, deprivation, smoking status, alcohol intake, body mass index, pre-existing medical co-morbidities, and concurrent medication. Measures of performance (prediction errors, calibration and discrimination) will be determined in the test data for men and women separately and by ten-year age group. For children, descriptive statistics will be undertaken if there are currently too few serious events to allow development of a risk model. The final model will be externally evaluated in (a) geographically separate practices and (b) other relevant datasets as they become available.

Ethics and dissemination

The project has ethical approval and the results will be submitted for publication in a peer-reviewed journal.

Strengths and limitations of the study

  • The individual-level linkage of general practice, Public Health England testing, Hospital Episode Statistics and Office of National Statistics death register datasets enable a robust and accurate ascertainment of outcomes

  • The models will be trained and evaluated in population-representative datasets of millions of individuals

  • Shielding for clinically extremely vulnerable was advised and in place during the study period, therefore risk predictions influenced by the presence of some ‘shielding’ conditions may require careful consideration

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.06.28.20141986: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: Thank you for sharing your code.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The strengths and limitations of the approach have already been discussed in detail 8,11,23,24,41,42. In summary, key strengths include size, wealth of data on risk factors, good ascertainment of outcomes through multiple record linkage, prospective recording of outcomes, use of an established validated database which has been used to develop many risk prediction tools, and lack of selection, recall and respondent bias and robust analysis. UK general practices have good levels of accuracy and completeness in recording clinical diagnoses and prescribed medications 43. We think our study has good face validity since it has been conducted in the setting where most patients in the UK are assessed, treated and followed up. Limitations: Limitations of our study include the lack of formal adjudication of diagnoses, potential for misclassification of outcomes depending on testing, information bias, and potential for bias due to missing data. Our database has linked mortality and hospital admissions data and is therefore likely to have picked up the great majority of COVID-19 related ICU admissions and death thereby minimising ascertainment bias. The initial evaluation will be done on a separate set of practices and individuals to those which were used to develop the score although the practices all use the same GP clinical computer system (EMIS – the computer system used by 55% of UK GPs). An independent evaluation will be a more stringent test and should be done (e.g. using data fro...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.06.28.20141986v1 on medRxiv