Lessons from a rapid systematic review of early SARS-CoV-2 serosurveys

  1. Niklas Bobrovitz
  2. Rahul Krishan Arora
  3. Tingting Yan
  4. Hannah Rahim
  5. Nathan Duarte
  6. Emily Boucher
  7. Jordan Van Wyk
  8. Timothy Grant Evans

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background. As the world grapples with the COVID-19 pandemic, there is increasing global interest in the role of serological testing for population monitoring and to inform public policy. However, limitations in serological study designs and test standards raise concerns about the validity of seroprevalence estimates and their utility in decision-making. There is now a critical window of opportunity to learn from early SARS-CoV-2 serology studies. We aimed to synthesize the results of SARS-CoV-2 serosurveillance projects from around the world and provide recommendations to improve the coordination, strategy, and methodology of future serosurveillance efforts. Methods. This was a rapid systematic review of cross-sectional and cohort studies reporting seroprevalence outcomes for SARS-CoV 2. We included completed, ongoing, and proposed serosurveys. The search included electronic databases (PubMed, MedRXIV, BioRXIV, and WHO ICTPR); five medical journals (NEJM, BMJ, JAMA, The Lancet, Annals of Internal Medicine); reports by governments, NGOs, and health systems; and media reports (Google News) from December 1, 2019 to May 1, 2020. We extracted data on study characteristics and critically appraised prevalence estimates using Joanna Briggs Institute criteria. Results. Seventy records met inclusion criteria, describing 73 studies. Of these, 23 reported prevalence estimates: eight preprints, 14 news articles, and one government report. These studies had a total sample size of 35,784 and reported 42 prevalence estimates. Seroprevalence estimates ranged from 0.4% to 59.3%. No estimates were found to have a low risk of bias (43% high risk, 21% moderate risk, 36% unclear). Fifty records reported characteristics of ongoing or proposed serosurveys. Overall, twenty countries have completed, ongoing, or proposed serosurveys. Discussion. Study design, quality, and prevalence estimates of early SARS-CoV2 serosurveys are heterogeneous, suggesting that the urgency to examine seroprevalence may have compromised methodological rigour. Based on the limitations of included studies, future serosurvey investigators and stakeholders should ensure that: i) serological tests used undergo high-quality independent evaluations that include cross-reactivity; ii) all reports of serosurvey results, including media, describe the test used, sample size, and sampling method; and iii) initiatives are coordinated to prevent test fatigue, minimize redundant efforts, and encourage better study methodology. Other. PROSPERO: CRD42020183634. No third-party funding.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.05.10.20097451: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    To conduct this “living” rapid review, we used abbreviated systematic review methods informed by Cochrane guidance.
    Cochrane guidance
    suggested: (ChiCTR - Chinese Clinical Trial Registry, RRID:SCR_006037)
    Search Strategy: A rapid systematic review was undertaken, searching for published and unpublished SARS-CoV-2 serosurveys from December 1, 2019 to May 1, 2020 in: electronic databases (PubMed, MedRXIV, BioRXIV, and WHO ICTPR); high-impact medical journals (NEJM, BMJ, JAMA, The Lancet, Annals of Internal Medicine); reports by governments, NGOs, and health systems; and media reports (Google News).
    PubMed
    suggested: (PubMed, RRID:SCR_004846)
    BioRXIV
    suggested: (bioRxiv, RRID:SCR_003933)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitations: This review had some limitations. Firstly, it is possible that articles were missed by only searching one academic database of peer-review articles. That said, the supplemental search included five high-impact journals, two pre-print databases, and a trial registry. Secondly, we did not conduct article screening or extraction using two independent authors. However, we pilot-tested screening and extraction in duplicate to strengthen reliability. Furthermore, a second author verified screening decisions and extracted data. Conclusions: The world is entering the next phase of the SARS-CoV-2 pandemic - attempting a return to normalcy. The ability to accurately map seroprevalence patterns will be a key feature of this phase as scientists determine the relationship between antibody levels and immunity, and as decision-makers consider policies to ease restrictions on movement and reopen economies. We should enter this phase armed with the lessons from early serosurveys: namely, that we need to raise the bar on seroprevalence testing initiatives and we need to do it together.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.05.10.20097451 on medRxiv