Sensitivity analysis of the effects of non-pharmaceutical interventions on COVID-19 in Europe
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
The role of non-pharmaceutical interventions (NPIs) on the spread of SARS-CoV-2 has drawn significant attention, both scientific and political. Particularly, an article by the Imperial College COVID-19 Response Team (ICCRT), published online in Nature on June 8, 2020, evaluates the efficiency of 5 NPIs. Based on mortality data up to early May, it concludes that only one of the interventions, lockdown, has been efficient in 10 out of 11 studied European countries.
We show, via simulations using the ICCRT model code, that conclusions regarding the effectiveness of individual NPIs are not justified. Our analysis focuses on the 11th country, Sweden, an outlier in that no lockdown was effectuated. The new simulations show that estimated NPI efficiencies across all 11 countries change drastically unless the model is adapted to give the Swedish data special treatment. While stated otherwise in the Nature article, such adaptation has been done in the model code reproducing its results: An ungrounded country-specific parameter said to have been introduced in all 11 countries, is in the code only activated for Sweden. This parameter de facto provides a new NPI category, only present in Sweden, and with an impact comparable to that of a lockdown.
While the considered NPIs have unarguably contributed to reduce virus spread, our analysis reveals that their individual efficiency cannot be reliably quantified by the ICCRT model, provided mortality data up to early May.
Article activity feed
-
SciScore for 10.1101/2020.06.15.20131953: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:We found the combination of high model sensitivity, and the assumption of Rt being driven solely by the NPIs, to constitute a fundamental limitation. This should be considered when the modelling results are used as a basis for policymaking. The perhaps most remarkable result in [4] is that almost all the decrease of viral reproduction …
SciScore for 10.1101/2020.06.15.20131953: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:We found the combination of high model sensitivity, and the assumption of Rt being driven solely by the NPIs, to constitute a fundamental limitation. This should be considered when the modelling results are used as a basis for policymaking. The perhaps most remarkable result in [4] is that almost all the decrease of viral reproduction was attributed to the lockdown intervention in the 10 countries where it was effectuated. In those same 10 countries, the effects of the 5 other considered intervention categories were almost negligible. In the one country where lockdown was not effectuated (Sweden), it was instead the public events ban intervention that almost alone contributed to attaining Rt<1. It seems unlikely that this would be the result of fortunate circumstances, where lockdown was implemented in exactly the 10 countries where it would have a large impact on Rt, and omitted in the single country, where it was not needed. The distribution modelling the infection-to-death delay in [4] has a mean of 23.9 days, and so the model could not predict this outcome at the time most countries chose to effectuate lockdown, while one country did not. An arguably more plausible explanation of the findings is that it is the last of all interventions, that in the model will contribute to bringing Rt<1 in a specific country. This is confirmed by executing the model [9] with different interventions being defined as having occurred last in the one country where lockdown was not effectuated...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
