Testing behaviour may bias observational studies of vaccine effectiveness

  1. Paul Glasziou
  2. Kirsten McCaffery
  3. Erin Cvejic
  4. Carys Batcup
  5. Julie Ayre
  6. Kristen Pickles
  7. Carissa Bonner

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

BACKGROUND: Recent observational studies suggest that vaccines may have little effect in preventing infection with the Omicron variant of severe acute respiratory syndrome coronavirus 2. However, the observed effects may be confounded by patient factors, preventive behaviours, or differences in testing behaviour. To assess potential confounding, we examined differences in testing behaviour between unvaccinated and vaccinated populations. METHODS: We recruited 1,526 Australian adults for an online randomized study about coronavirus disease 2019 (COVID-19) testing in late 2021, collecting self-reported vaccination status and three measures of COVID-19 testing behaviour: testing in past month or ever and test intention if they woke with a sore throat. We examined the association between testing intentions and vaccination status in the trial’s baseline data. RESULTS: Of the 1,526 participants (mean age 31 y), 22% had a COVID-19 test in the past month and 61% ever; 17% were unvaccinated, 11% were partially vaccinated (one dose), and 71% were fully vaccinated (two or more doses). Fully vaccinated participants were twice as likely as those who were unvaccinated (relative risk [RR] 2.2, 95% CI 1.8 to 2.8, p < 0.001) to report positive COVID testing intentions. Partially vaccinated participants had less positive intentions than fully vaccinated participants (RR 0.68, 95% CI 0.52 to 0.89, p < 0.001) but higher intentions than unvaccinated participants (RR 1.5, 95% CI 1.4 to 1.6, p = 0.002). DISCUSSION: Vaccination predicted greater COVID-19 testing intentions and would substantially bias observed vaccine effectiveness. To account for differential testing behaviours, test-negative designs are currently the preferred option, but their assumptions need more thorough examination.

Article activity feed

  1. Version published to 10.3138/jammi-2022-0002
  2. ScreenIT

    SciScore for 10.1101/2022.01.17.22269450: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variableWe recruited 1,526 Australian adults (430 men, 1064 women, 32 non-binary or not reported) for an online randomised study about COVID testing between October and November 2021 (www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=382318) using methods we have detailed previously6.
    RandomizationWe recruited 1,526 Australian adults (430 men, 1064 women, 32 non-binary or not reported) for an online randomised study about COVID testing between October and November 2021 (www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=382318) using methods we have detailed previously6.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2022.01.17.22269450v1 on medRxiv