Syndromic Surveillance-Based Estimates of Vaccine Efficacy Against COVID-Like Illness from Emerging Omicron and COVID-19 Variants

  1. Tanner J. Varrelman
  2. Benjamin Rader
  3. Christina M. Astley
  4. John S. Brownstein

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

New infections from the omicron variant of SARS-CoV-2 have been increasing dramatically in South Africa since first identification in November 2021. Despite increasing uptake of COVID-19 vaccine, there are concerns vaccine protection against omicron may be reduced compared to other variants. We sought to characterize a surrogate measure of vaccine efficacy in Gauteng, South Africa by leveraging real-time syndromic surveillance data. The University of Maryland Global COVID Trends and Impact Survey (UMD-CTIS) is an online, cross-sectional survey conducted among users sampled from the Facebook active user base. We derived three COVID-like illness (CLI) definitions (stringent, classic, and broad) using combinations of self-reported symptoms (present or not in the prior 24 hours) that broadly tracked with reported COVID-19 cases during June 18, 2021 - December 14, 2021 (inclusive of the delta wave and up-trend of the omicron wave). We used syndromic-surveillance-based CLI prevalence measures among the vaccinated ( P V ) and unvaccinated ( P U ) respondents to estimate V E CLIP = 1 - ( P V / P U ), a proxy for vaccine efficacy, during the delta (June 18-July 18, N= 9,387 surveys) and omicron (December 4-14, N= 2,389 surveys) wave periods. We assume no waning immunity, CLI prevalence approximates incident infection with each variant, and vaccinated and unvaccinated survey respondents in the two variant wave periods are exchangeable. The vaccine appears to have consistently lower V E CLIP against omicron, compared to delta, regardless of the CLI definition used. Stringent CLI (i.e. anosmia plus fever, cough and/or myalgias) yielded a delta V E CLIP = 0.85 [0.54, 0.95] higher than omicron V E CLIP = 0.62 [0.46, 0.72]. Classic CLI (cough plus anosmia, fever, and/or myalgias) gave lower estimates (delta V E CLIP = 0.76 [0.54, 0.87], omicron V E CLIP = 0.51 [0.42, 0.59]), but omicron was still lower than delta. We acknowledge the potential for measurement, confounding, and selection bias, as well as limitations for generalizability for these self-reported, syndromic surveillance-based V E CLIP measures. Thus V E CLIP as estimates of true, population-level vaccine efficacy should therefore be taken with caution. Nevertheless, these preliminary findings demonstrating declining V E CLIP raise concern for a true decline in vaccine efficacy versus waning immunity as a potential contributor to the omicron variant taking hold in Gauteng and elsewhere.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2021.12.17.21267995: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    UMD-CTIS is a statistical sample of Facebook users, and there may be important limitations to generalizing effect estimates from a selected sample [13]. Further studies could address this by incorporating covariates – such as gender, age, illness duration, and the potential risk behaviors of survey respondents – to fully understand the potential biases, and better characterize the potential risks of emerging virus variants. While the syndromic surveillance informed proxy of vaccine efficacy may not be the gold standard for estimating vaccine efficacy, we have shown that analysis of regional data from UMD-CTIS can quickly point towards high-level changes leveraging remote, real-time CLI trends survey respondents across time. Still, this initial report has only described a very limited set of questions that can be asked using UMD-CTIS to better understand the epidemiological profile of Omicron in Gauteng, South Africa and globally. As we continue to face SARS-CoV-2 variants, and seasonal and emerging infectious disease in general, it is crucial that we combine insights from lab-based studies, with early insights from syndromic surveillance data streams.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2021.12.17.21267995 on medRxiv