Utility of COVID-19 Decision Rules Related to Consecutive Decline in Positivity or Hospitalizations: A Data-driven Simulation Study

  1. Keisuke Ejima
  2. Kevin Maki
  3. Lilian Golzarri-Arroyo
  4. David B. Allison

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

The White House issued Guidelines for Opening Up America Again to help state and local officials when reopening their economies. These included a 'downward trajectory of positive tests as a percent of total tests within a 14-day period.' To examine this rule, we computed the probability of observing continuous decline in positivity when true positivity is in decline using data-driven simulation. Data for COVID-19 positivity reported in New York state from April 14 to May 5, 2020, where a clear reduction was observed, were used. First, a logistic regression model was fitted to the data, considering the fitted values as true positivity. Second, we created observed positivity by randomly selecting 25,000 people per day from a population with those true positivity for 14 days. The simulation was repeated 1,000 times to compute the probability of observing a consecutive decline. As sensitivity analyses, we performed the simulation with different daily numbers of tests (10 to 30,000) and length of observation (7 and 21 days). We further used daily hospitalizations as another metric, using data from the state of Indiana. With 25,000 daily tests, the probability of a consecutive decline in positivity for 14 days was 99.9% (95% CI: 99.7% to 100%). The probability dropped with smaller numbers of tests and longer lengths of consecutive observation, because there is more chance of observing an increase in positivity with smaller numbers of tests and longer observation. The probability of consecutive decline in hospitalizations was ~0.0% regardless of the length of consecutive observation due to large variance. These results suggest that continuous declines in sample COVID-19 test positivity and hospitalizations may not be observed with sufficient probability, even when population probabilities truly decline. Criteria based on consecutive declines in metrics are unlikely to be useful for making decisions about relaxing COVID-19 mitigation efforts.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.12.14.20248190: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    All simulations were performed using the statistical computing software R 4.0.1 (R Development Core Team).
    R Development Core
    suggested: (R Project for Statistical Computing, RRID:SCR_001905)

    Results from OddPub: Thank you for sharing your data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    A limitation of this study is that we assumed logistic or linear models for the outcome metrics. Numerous mathematical models have been proposed to describe COVID-19 infection dynamics (15-17), some of which might better explain the COVID-19 dynamics from the beginning to the end. However, we did not employ those models because we focused only on the declining phase of the epidemic, which can be described by those simple logistic and linear models without losing generalizability. If the actual disease dynamics fluctuate when the epidemic is moving toward the containment phase, the probability of consecutive decline in observed outcomes might decrease further than what we observed in our simulation. Compared with the criterion based on consecutive decline metrics, the effective reproduction number is more robust against such measurement error because its computation is dependent on the longitudinal case reports weighted by an infectivity function, rather than the number of cases reported on a single time point. Further, the fluctuation of the effective reproduction number has been observed and acknowledged (7, 18). Therefore, researchers are advised to assume the effective reproduction number is constant for a short period to simplify interpretation. There is a long history for conceptualization and computation of the effective reproduction number, whereas consecutive decline has not been examined until our study, as far as we know.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.12.14.20248190v1 on medRxiv