Accuracy of the Veterans Health Administration COVID-19 (VACO) Index for predicting short-term mortality among 1307 US academic medical centre inpatients and 427 224 US Medicare patients

  1. Joseph T King, Jr.
  2. James S Yoon
  3. Zachary M Bredl
  4. Joseph P Habboushe
  5. Graham A Walker
  6. Christopher T Rentsch
  7. Janet P Tate
  8. Nitu M Kashyap
  9. Richard C Hintz
  10. Aneesh P Chopra
  11. Amy C Justice

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The Veterans Health Administration COVID-19 (VACO) Index predicts 30-day all-cause mortality in patients with COVID-19 using age, sex and pre-existing comorbidity diagnoses. The VACO Index was initially developed and validated in a nationwide cohort of US veterans—we now assess its accuracy in an academic medical centre and a nationwide US Medicare cohort.

Methods

With measures and weights previously derived and validated in US national Veterans Health Administration (VA) inpatients and outpatients (n=13 323), we evaluated the accuracy of the VACO Index for estimating 30-day all-cause mortality using area under the receiver operating characteristic curve (AUC) and calibration plots of predicted versus observed mortality in inpatients at a single US academic medical centre (n=1307) and in Medicare inpatients and outpatients aged 65+ (n=427 224).

Results

30-day mortality varied by data source: VA 8.5%, academic medical centre 17.5%, Medicare 16.0%. The VACO Index demonstrated similar discrimination in VA (AUC=0.82) and academic medical centre inpatient population (AUC=0.80), and when restricted to patients aged 65+ in VA (AUC=0.69) and Medicare inpatient and outpatient data (AUC=0.67). The Index modestly overestimated risk in VA and Medicare data and underestimated risk in Yale New Haven Hospital data.

Conclusions

The VACO Index estimates risk of short-term mortality across a wide variety of patients with COVID-19 using data available prior to or at the time of diagnosis. The VACO Index could help inform primary and booster vaccination prioritisation, and indicate who among outpatients testing positive for SARS-CoV-2 should receive greater clinical attention or scarce treatments.

Article activity feed

  1. Version published to 10.1136/jech-2021-216697
  2. ScreenIT

    SciScore for 10.1101/2021.01.01.20249069: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Data analyses were performed using Stata, version 15.1 (StataCorp, College Station, TX) and SAS 9.4 (SAS Institute Inc, Cary, NC).
    StataCorp
    suggested: (Stata, RRID:SCR_012763)
    SAS Institute
    suggested: (Statistical Analysis System, RRID:SCR_008567)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    There are limitations to our study. While we were able to evaluate the index in a national sample of over 427,000 people over the age of 65, our ability to validate in substantially younger patients remains somewhat limited. None of the Medicare sample, only 25% of the VA sample, and 20% of the YNHH samples were under 50 years of age. However, the most pressing questions concerning vaccine allocation are focused on individuals over 50 years of age. Also, our calibration is based on mortality events occurring before August 2020 when mostly symptomatic patients were being tested. The VACO Index may over-estimate risk of mortality among asymptomatic patients and this will require further study. Nevertheless, the VACO Index represents the most widely validated risk index for short term mortality from COVID-19. This standardized and validated index can supplement clinical judgement to help inform patient management and health policy.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.01.01.20249069v1 on medRxiv