Effectiveness of non-pharmaceutical interventions to contain COVID-19: a case study of the 2020 spring pandemic wave in New York City

  1. Wan Yang
  2. Jaimie Shaff
  3. Jeffrey Shaman

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

As COVID-19 continues to pose significant public health threats, quantifying the effectiveness of different public health interventions is crucial to inform intervention strategies. Using detailed epidemiological and mobility data available for New York City and comprehensive modelling accounting for under-detection, we reconstruct the COVID-19 transmission dynamics therein during the 2020 spring pandemic wave and estimate the effectiveness of two major non-pharmaceutical interventions—lockdown-like measures that reduce contact rates and universal masking. Lockdown-like measures were associated with greater than 50% transmission reduction for all age groups. Universal masking was associated with an approximately 7% transmission reduction overall and up to 20% reduction for 65+ year olds during the first month of implementation. This result suggests that face covering can substantially reduce transmission when lockdown-like measures are lifted but by itself may be insufficient to control SARS-CoV-2 transmission. Overall, findings support the need to implement multiple interventions simultaneously to effectively mitigate COVID-19 spread before the majority of population can be protected through mass-vaccination.

Article activity feed

  1. Version published to 10.1098/rsif.2020.0822
  2. Version published to 10.1101/2020.09.08.20190710v2 on medRxiv
  3. NCRC

    Our take

    This preprint, which has not yet been subjected to peer review, used both case counts and deaths in an age-specific model of SARS-CoV-2 transmission in New York City. The authors estimate that the reproduction number of the virus decreased dramatically from the beginning of March to mid-April, 2020 in response to a collection of interventions that reduced mobility (through stay-at-home orders, school closures, business closures, etc.) and mandated mask use. The authors’ attribution of transmission declines to each of these two types of interventions, however, is dependent on many assumptions and subject to much uncertainty.

    Study design

    modeling-simulation;other

    Study population and setting

    This study estimated the impacts on SARS-CoV-2 transmission of non-pharmaceutical interventions (NPIs) in New York City (NYC) from March 1 to June 6, 2020. COVID-19 cases included all laboratory-confirmed cases reported to the NYC Department of Health and Mental Hygiene, and deaths combined probable and confirmed deaths associated with COVID-19. Mobility data, used as a proxy for contact rates, were obtained from Safegraph and consisted of anonymized, aggregated counts of visitors (measured by mobile phone location) to locations within each ZIP code. The authors used a neighborhood-specific SEIR network model fit to cases and deaths, stratified by age group, to estimate the effects on transmission of 1) all NPIs, 2) contact-reducing NPIs (such as stay-at-home orders, business closures, etc.), and 3) mask use. Mask use was assumed to explain the reduction in estimated transmission rate that was not accounted for by mobility declines during periods when face coverings were mandated in public places. Model projections beyond the end of the study period were compared to observed cases and deaths.

    Summary of main findings

    Observed, diagnosed COVID-19 cases displayed different age-specific patterns compared to model estimates of underlying infection rates: estimated infection rates were highest for those aged 25-44 years and 45-64 years, and rates for all age groups peaked the week of March 22 or one week later. During the first week of the NYC epidemic (beginning March 1), the estimated time-varying reproduction number (Rt) was 2.99, decreased to 1.37 after the stay-at-home order on March 22, and reached a minimum of 0.56 during the week of April 12. Mobility reductions (a proxy for contact rate reductions arising from stay-at-home mandates, school closures, and other contact-reducing interventions) were estimated to result in a 70.7% (95% CI: 65.0% to 76.4%) decline in Rt by the week of April 12. Assuming that effectiveness of mask use would account for the difference between estimates using a) a linear regression with mobility data alone and b) the SEIR model, the authors estimated that mask use reduced the transmission rate and infectious period by 3.4% (95% CI: -1.9% to 8.6%) over eight weeks, with higher effectiveness during the first month. Estimated mask effectiveness was highest in older age groups and remained stable during the study period (for the first month among those 65-74 years old: 20.8%, 95% CI: -0.1 to 41.6%; 75+ years old: 20.8%, 95% CI: 20.8%, 95% CI: -0.9 to 42.5%). Projections from the week of June 7 through the week of July 26 using parameters based on observed mobility data and estimated mask effectiveness underestimated cumulative cases by 27% and underestimated deaths by 2%.

    Study strengths

    The model was fit to both observed cases and deaths, and projections beyond the study period were compared to observed outcomes.

    Limitations

    This is a preprint, and has not yet been subject to peer review. Aggregated zip-code-level mobility data are an imperfect proxy for actual mobility, which is in turn an imperfect proxy for contact rates. The method used to estimate the effectiveness of mask use relies on strong assumptions and oversimplifications (e.g., all residual reduction in predicted transmission rate after accounting for mobility decline is attributable to mask use; mask use affects both transmission risk per contact and infectious period; dates of mask mandates are a perfect proxy for actual mask use; etc.). Projections did not fit observed data well, which may be an indication that the effect of interventions was overestimated.

    Value added

    This study provides a useful picture of age-specific patterns of SARS-CoV-2 infection during the spring of 2020 in New York City.

  4. ScreenIT

    SciScore for 10.1101/2020.09.08.20190710: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    In addition, SafeGraph also provided an aggregate measure of the length of time spent outside of the home during each week.
    SafeGraph
    suggested: None

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    We also note there remain large uncertainties in our estimates due to several limitations. First, we used population mobility as a proxy for contact rates rather than more direct measures. Similar approximation and uncertainty applied to our estimates of the effectiveness of face covering. Future studies are thus warranted for further assessment. For instance, large population scale surveys documenting changes of the intensity and pattern of contact during the pandemic could provide more accurate measures of contact rates among different age groups and over time. Second, while we restricted our analysis on the effectiveness of face covering to a period when masks were mandated, there remain other residual confounding effects. For instance, increased awareness of COVID-19 and health risk among key age groups such as the elderly may have contributed to further reductions of transmission through other precautions in addition to face covering; this may have led to an overestimation of the effectiveness of face covering for those age groups. Third, here we focused on estimating the effectiveness of interventions in the general population without segregating key settings with intense transmission (e.g., long-term care facilities). Future studies should assess the impact of interventions targeting such high-risk settings. Lastly, our estimates here were largely based on the first wave of the pandemic and may not fully capture subsequent changes in awareness and perception of COVID-1...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  5. Version published to 10.1101/2020.09.08.20190710v1 on medRxiv