Estimating the effectiveness of the Pfizer COVID-19 BNT162b2 vaccine after a single dose. A reanalysis of a study of ‘real-world’ vaccination outcomes from Israel
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
A distinctive feature of the roll out of vaccination against SARS-CoV-2 virus in the UK was the decision to delay the timing of the second injection till 12 weeks after the first. The logic behind this is to protect more people sooner and so reduce the total number of severe infections, hospitalisations, and deaths. This decision caused criticism from some quarters due in part to a belief that a single injection may not give adequate immunity. A recent paper based on Israel’s experience of vaccination suggested that a single dose may not provide adequate protection. Here we extract the primary data from the Israeli paper and then estimate the incidence per day for each day after the first injection and also estimate vaccine effectiveness for each day from day 13 to day 24. We used a pooled estimate of the daily incidence rate during days 1 to 12 as the counterfactual estimate of incidence without disease and estimated confidence intervals using Monte Carlo modelling. After initial injection case numbers increased to day 8 before declining to low levels by day 21. Estimated vaccine effectiveness was pretty much 0 at day 14 but then rose to about 90% at day 21 before levelling off. The cause of the initial surge in infection risk is unknown but may be related to people being less cautious about maintaining protective behaviours as soon as they have the injection. What our analysis shows is that a single dose of vaccine is highly protective, although it can take up to 21 days to achieve this. The early results coming from Israel support the UK policy of extending the gap between doses by showing that a single dose can give a high level of protection.
Article activity feed
-
SciScore for 10.1101/2021.02.01.21250957: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization Both Actual and Expected case numbers were randomly estimated 10,000 times using the Binomial function. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:There are a number of weaknesses in the analyses undertaken by the authors of the Chodick et al paper that need to be addressed. Firstly, …
SciScore for 10.1101/2021.02.01.21250957: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization Both Actual and Expected case numbers were randomly estimated 10,000 times using the Binomial function. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:There are a number of weaknesses in the analyses undertaken by the authors of the Chodick et al paper that need to be addressed. Firstly, the reported effectiveness included data from days (13 to 17) when it was apparent that the vaccine was not yet providing much protection and therefore their analyses could not provide any real estimate of the overall effectiveness of a single dose when the second dose was delayed for up to 12 weeks. By analysing effectiveness for each day, the analysis approach presented here overcomes this weakness. Secondly, by comparing incidence in a later period with that in an earlier period, Chodick et al effectively used a historical rather than a contemporary control group. Whist such historical comparisons can have value they are known to have serious issues; it has been said that “most historical control groups are compromised for some reason” (Streiner & Norman 1998; Baker & Lindeman 2001). Indeed, in one of our own systematic reviews of dengue control strategies those studies using historical controls gave much greater apparent effectiveness (Al-Muhandis & Hunter 2011). In the context of this study, a key issue is whether individuals after their first dose might start to be less careful about social distancing and so increase their exposure risk (SPI-B 2020). Whilst it is too early to have empirical data on this, the finding that incidence increased for a week after first injection could be an indication that such increased exposure does happe...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
