Article activity feed

  1. Our take

    This study, available as a preprint and thus not yet peer reviewed, used data from nationwide registries in the Netherlands to estimate vaccine effectiveness against COVID-19 hospitalization during periods when the Alpha and Delta variants were dominant. The study found that full vaccination (from the Pfizer, Moderna, AstraZeneca, and Janssen vaccines) was highly effective in both periods (94% and 95%, respectively), and found no evidence that effectiveness waned over a 5-month span after full vaccination or across different age groups. Although there are some possible sources of bias in this study, the use of nationwide registries provides strong evidence for persistently robust protection of vaccination against COVID-19-associated hospitalization, even with the dominance of the Delta strain.

    Study design


    Study population and setting

    This study in the Netherlands linked a nationwide vaccination registry with a nationwide hospitalization database to estimate vaccine effectiveness (VE) against COVID-19 hospitalization and ICU admission from April 4 to August 19, 2021. VE was estimated by vaccine (Pfizer, Moderna, AstraZeneca, and Janssen), age group (15-49 years, 50-69 years, 70 years and older), time since vaccination, and prevailing SARS-CoV-2 variant (April 4 to May 29, when 95% of sequenced SARS-CoV-2 isolates were Alpha, vs. July 4 - August 29, when 99% of sequenced isolates were Delta). COVID-19 hospitalizations and ICU admissions were taken from NICE, a nationwide registry of all hospitalized individuals with a positive SARS-CoV-2 test or COVID-19 diagnosis. Vaccination status was ascertained via the nationwide vaccine registry CIMS. Upon vaccination, informed consent was sought for CIMS registration; among the 84% of vaccinations provided through the Netherlands municipal health services, 7.3% declined to be registered in CIMS. The size of the unvaccinated group was calculated by subtracting the vaccinated group from the full population. Vaccination status was considered to be “partial” 14 days after the first dose and “full” 14 days after the second dose (or 28 days after the single dose of the Janssen vaccine). The median duration from symptom onset to hospitalization, by age group (3-7 days), was applied to each admission date to classify vaccination status. Incidence rates per 10,000 person-days were calculated and incidence rate ratios (IRRs) were calculated with negative binomial regression adjusted for calendar date.

    Summary of main findings

    Of the 15,571 patients hospitalized with COVID-19 during the study period, 6% were fully vaccinated, 7% were partially vaccinated, and 87% were unvaccinated. For full vaccination, the estimated VE against hospitalization during the Alpha period was 94% overall (95% CI: 93% to 95%) and was consistently high across all three age groups. The estimated VE against hospitalization during the Delta period was 95% (94% to 95%) and also above 90% across age groups. Estimated VE against hospitalization by vaccine during the Delta period was as follows: Pfizer 96% (95% to 96%), Moderna 84% (80% to 87%), AstraZeneca 94% (92% to 95%), and Janssen 91% (88% to 94%). Estimated VE against ICU admission was generally slightly higher than that for hospitalization across categories of age, vaccine, time since vaccination, and viral variant period. There was no evidence of waning VE in any age group up to 20 weeks after full vaccination.

    Study strengths

    Linkage of national hospitalization and vaccination registries allowed for a full nationwide cohort analysis of vaccine effectiveness and a comparison of vaccine effectiveness across age and vaccine cohorts.


    Vaccine allocation was nonrandom; for example, the authors noted that the Moderna vaccine was provided to medically high-risk patients, which may explain the lower apparent VE for that vaccine relative to the other vaccines. Furthermore, individuals who received the Pfizer and Moderna vaccine received the doses with a median interval of five weeks, which is longer than the intervals tested in their clinical trials (three and four weeks respectively), but may have implications for long-term effectiveness. Additionally, other than age, no information on patient characteristics or comorbidities were available, so unmeasured confounding is likely. There were few hospitalizations in several strata, limiting precision of estimates. An unknown proportion (likely around 7%) of vaccinated individuals did not consent to CIMS registration, and the resulting misclassification would result in overestimation of VE. However, a sensitivity analysis showed minimal impact on VE estimates. The population at risk included people with prior SARS-CoV-2 infection, which may have also led to bias if prior infection was associated with both vaccination status and hospitalization. Finally, COVID-19 hospitalization may have included some people who were hospitalized for other reasons but who incidentally tested positive for SARS-CoV-2, which may have led to a slight bias in an unknown direction.

    Value added

    This study is one of the first nationwide cohort studies to estimate vaccine effectiveness in the era of dominance of the Delta strain of SARS-CoV-2.

    Read the original source
    Was this evaluation helpful?
  2. SciScore for 10.1101/2021.09.15.21263613: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    However, this dataset has three important limitations. Firstly, the NICE COVID-19 dataset does not contain information on comorbidities. When interpreting our results, it is important to consider that groups with increased risk of severe COVID-19 were prioritized for vaccination in early 2021. The relatively low VE of Spikevax (Moderna) might be related to the use of specifically this vaccine for patient groups at medically high risk for severe COVID-19, such as immunocompromised patients. In this group, the chance of vaccine breakthrough infection requiring hospitalization is relatively high (12-14). Other studies have not found a lower VE against severe COVID-19 for the Spikevax vaccine in the general population. Puranik et al. found a Spikevax VE against hospitalization of 81% in the US in July 2021, and a Canadian study found a single-dose Spikevax VE against hospitalization with Delta of 96% (6, 15). Secondly, the NICE COVID-19 registry includes all hospitalized patients with a positive SARS-CoV-2 test, also when the indication for hospitalization was not COVID-19, when patients test positive by routine screening at admission. This might bias our VE estimates if vaccination impacts the likelihood of being admitted with SARS-CoV-2 rather than because of SARS-CoV-2. Such misclassification of COVID-19 hospitalizations could be more likely for vaccinated patients as their risk of severe COVID-19 is greatly reduced resulting in underestimation of the VE. On the other hand unv...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

    Read the original source
    Was this evaluation helpful?