The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories

Article activity feed

1. Version published to 10.1186/s12916-020-01872-8

   Feb 5, 2021
2. 

   NCRC

   Oct 20, 2020

   Our take

   This study, available as a preprint and thus not yet peer reviewed, evaluated the impact of 13 different non-pharmaceutical interventions on the time varying reproductive number across 130 countries and territories through June 2020. Impact was assessed using a wide range of models with different time lags (i.e., time frame from implementation of intervention to its impact on transmission) and at varying levels of intensity of the interventions. School closures and internal movement restrictions showed evidence of impact across all models, and thus had strong evidence of effectiveness. However, these results should be interpreted cautiously as multiple interventions were often implemented at the same time, making it difficult to fully disentangle the impact of any single intervention from the others.

   Study …

   Our take

   This study, available as a preprint and thus not yet peer reviewed, evaluated the impact of 13 different non-pharmaceutical interventions on the time varying reproductive number across 130 countries and territories through June 2020. Impact was assessed using a wide range of models with different time lags (i.e., time frame from implementation of intervention to its impact on transmission) and at varying levels of intensity of the interventions. School closures and internal movement restrictions showed evidence of impact across all models, and thus had strong evidence of effectiveness. However, these results should be interpreted cautiously as multiple interventions were often implemented at the same time, making it difficult to fully disentangle the impact of any single intervention from the others.

   Study design

   ecological;modeling-simulation

   Study population and setting

   This study used the implementation dates of a large variety of non-pharmaceutical policy interventions and assed their relationship with COVID-19 time varying reproductive number, Rt, defined as the mean number of secondary cases that one index case will infect at time t) across 130 countries and territories between January 1 and June 22, 2020. Data on non-pharmaceutical COVID-19 policies, categories, implementation dates, and a general index of strength of COVID-19 policy response were obtained from the Oxford COVID-19 Government Response Tracker. Estimates of the Rt across regions and time were from EpiForecasts. There were four primary analyses. The first main analysis used a statistical regression model to characterize how different types of NPIs were rolled out over time, such as when they occurred and in what order. The second model observed how those NPIs were clustered in time (i.e. what policies were more likely to occur together with what other policies, forming a group or a cluster of policies). A third model attempted to determine the time lags between policy implementation and their effect on Rt by assessing 3 different time lags (1, 5, and 10 days) and estimating goodness-of-fit statistics for each model. Finally, the study used panel regression models to attempt to disentangle the impact of different types of NPIs on Rt accounting for different time lags and the effort at which NPI was implemented (any effort and maximum effort).

   Summary of main findings

   First, the study finds a major increase in NPI intensity across the world in mid-March, followed by a slow reduction in the stringency of interventions. Second, the study finds substantial evidence that NPI policies were more likely to be rolled out in particular orders, typically clustered together in time, depending in part on how policy intensity is defined (any effort or maximum effort). Thirdly, the study found support for lag times between policy implementation and Rt impact between 1-10 days. Finally, the study finds evidence that school closure and internal movement restrictions, and high-intensity public events cancellations and restrictions on gatherings. There was some evidence for impact for workplace closure, income support and debt/contract relief. Evidence for impact was inconclusive for stay-at-home requirements, public information campaigns, public transport closure, international travel controls, testing, and contact tracing.

   Study strengths

   This study critically examines and demonstrates how NPI policies are related to each other, and assesses lags in their impact. These are issues that are critical both to this analysis, and many other policy analyses which often do not acknowledge that NPI policies are highly correlated with respect to timing of implementation, and that interventions have lagging effects, due to changing compliance over time. The study uses well-vetted data appropriate for policy impact evaluation. The discussion section contains a frank and well-written interpretation of the results and the interpretable limitations thereof. We find that the findings that NPIs overall had substantial impact on Rt to be relatively robust.

   Limitations

   While the discussion section involves cautious interpretation of impact of individual NPI because they were implemented at similar times, the highest impact section (the abstract) does not heed that caution, and strongly implies that they identified which specific types of NPIs were most effective despite temporal correlation. The key difficulty - one examined and partially established in the paper itself - is that these interventions are temporally related to each other, and also have time lagged effects in similar timescales as the policy rollouts. One major issue is that there appears to be substantial limitations in the way in which lagged effects were measured. Lags were assumed to be at a fixed amount of time and the same for all policies, but in reality policy lag effects are spread out over a wide period of time, and will be different for different policies in different situations. This, in addition to the close clustering in time between policies, makes it difficult to conclude which NPIs had the strongest evidence for impact.

   Value added

   The study shows strong evidence that NPI policies are related to each other over time and how that creates difficulty in examining their impact.

   Read the original source
3. 

   ScreenIT

   Aug 12, 2020

   SciScore for 10.1101/2020.08.11.20172643: (What is this?)

   Please note, not all rigor criteria are appropriate for all manuscripts.

   Table 1: Rigor

   NIH rigor criteria are not applicable to paper type.

   Table 2: Resources

   No key resources detected.

   ---

   Results from OddPub: Thank you for sharing your code.

   ---

   Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:

   Nonetheless, our study also has several limitations. First, besides the information bias in the NPIs database discussed above, the coding scheme may also introduce potential bias. NPIs coded as “comprehensive contact tracing for all identified cases” may have different implications in different countries. Effectiveness of contact tracing in places like Singapore (39) may be masked by seemingly similar but realistic non-comparable contact …

   SciScore for 10.1101/2020.08.11.20172643: (What is this?)

   Please note, not all rigor criteria are appropriate for all manuscripts.

   Table 1: Rigor

   NIH rigor criteria are not applicable to paper type.

   Table 2: Resources

   No key resources detected.

   ---

   Results from OddPub: Thank you for sharing your code.

   ---

   Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:

   Nonetheless, our study also has several limitations. First, besides the information bias in the NPIs database discussed above, the coding scheme may also introduce potential bias. NPIs coded as “comprehensive contact tracing for all identified cases” may have different implications in different countries. Effectiveness of contact tracing in places like Singapore (39) may be masked by seemingly similar but realistic non-comparable contact tracing programs. Second, compared to daily incidence, Rt estimates are much more suitable to compare across countries and thus is used as the metric for COVID-19 transmission in this study. However, these estimates are based on a series of assumptions (e.g., distribution of onset to confirmation delay) that may not always be appropriate. Our current model also does not factor in uncertainty around Rt estimates. Last but not the least, although we examined a wide range of NPIs, we did not include any potential interactions in the current model. Such interaction is a possibility, e.g., more people may comply with workplace closures when receiving income support. Future research should look into these relationships.

   ---

   Results from TrialIdentifier: No clinical trial numbers were referenced.

   ---

   Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

   ---

   Results from JetFighter: We did not find any issues relating to colormaps.

   ---

   Results from rtransparent:

   • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
   • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
   • No protocol registration statement was detected.

   ---

   Read the original source
4. 

   Version published to 10.1101/2020.08.11.20172643v1 on medRxiv

   Aug 12, 2020