Evaluation of the performance of SARS‐­CoV­‐2 antibody assays for a longitudinal population­based study of COVID‐­19 spread in St. Petersburg, Russia

  1. Anton Barchuk
  2. Daniil Shirokov
  3. Mariia Sergeeva
  4. Rustam Tursun­zade
  5. Olga Dudkina
  6. Varvara Tychkova
  7. Lubov Barabanova
  8. Dmitriy Skougarevskiy
  9. Daria Danilenko

Reviewed by

Read the full article

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Geographical variation in severe acute respiratory syndrome coronavirus 2 (SARS­‐CoV­‐2) spread requires seroprevalence studies based on local tests, but robust validation is needed. We summarize an evaluation of antibody tests used in a serological study of SARS‐­CoV­‐2 in Saint Petersburg, Russia. We validated three different antibody assays: chemiluminescent microparticle immunoassay (CMIA) Abbott Architect SARS‐­CoV­‐2 immunoglobulin G (IgG), enzyme­ linked immunosorbent assay (ELISA) CoronaPass total antibodies test, and ELISA SARS­‐CoV‐­2‐­IgG‐­EIA‐­BEST. Clinical sensitivity was estimated with the SARS­‐CoV­‐2 polymerase chain reaction (PCR) test as the gold standard using manufacturer recommended cutoff. Specificity was estimated using pre­pandemic sera samples. The median time between positive PCR test results and antibody tests was 21 weeks. Measures of concordance were calculated against the microneutralization test (MNA).Sensitivity was equal to 91.1% (95% confidence intervbal [CI]: 78.8–97.5), 90% (95% CI: 76.4–96.4), and 63.1% (95% CI [50.2–74.7]) for ELISA Coronapass, ELISA Vector­Best, and CMIA Abbott, respectively. Specificity was equal to 100% for all the tests. Comparison of receiver operating characteristics has shown lower AUC for CMIA Abbott. The cut­off SC/O ratio of 0.28 for CMIA Abbott resulted in a sensitivity of 80% at the same level of specificity. Less than 33% of the participants with positive antibody test results had neutralizing antibodies in titers 1:80 and above. Antibody assays results and MNA correlated moderately. This study encourages the use of local antibody tests and sets the reference for seroprevalence correction. Available tests' sensitivity allows detecting antibodies within the majority of PCR­ positive individuals. The Abbott assay sensitivity can be improved by incorporating a new cut­off. Manufacturers' test characteristics may introduce bias into the study results.

Article activity feed

  1. Version published to 10.1002/jmv.27126
  2. ScreenIT

    SciScore for 10.1101/2021.04.05.21254712: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementConsent: Written informed consent was obtained in all cases of blood donation and further sample preparation.
    IRB: Ethical considerations and study registration: The study was approved by the Research Planning Board of European University at St. Petersburg (on May 20, 2020) and the Ethic Committee of the Clinic “Scandinavia” (on May 26, 2020).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.
    Cell Line Authenticationnot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    Antibody tests: We used three different antibody tests throughout the study: 1) chemiluminescent microparticle immunoassay Abbott Archi-tect SARS-CoV-2 IgG on the Abbott ARCHITECT® i2000sr platform (Abbott Laboratories, Chicago, USA), detecting immunoglobulin class G (IgG) antibodies to the nucleocapsid protein of SARS-CoV-2 with the signal/cut-off (S/CO) ratio of 1.4 for positivity (CMIA Abbot; www.fda.gov/media/137383/download).;
    immunoglobulin class G (IgG
    suggested: None
    2) enzyme-linked immunosorbent assay CoronaPass total antibodies test (Genetico, Moscow, Russia) based on recombinant receptor binding domain of the spike protein of SARS-CoV-2 (Department of Microbiology, Icahn School of Medicine at Mount Sinai, New York, NY, USA), detecting total antibodies with the S/CO ratio of 1.0 for positivity (ELISA Coronapass, pass.genetico.ru); 3) enzyme-linked immunosorbent assay SARS-CoV-2-IgG-EIA-BEST by Vector-Best, Novosibirsk, Russia also detecting IgG antibodies to the spike protein of SARS-CoV-2 with the S/CO ratio of 1.1 for positivity.
    Vector-Best, Novosibirsk, Russia also detecting IgG
    suggested: None
    Experimental Models: Cell Lines
    SentencesResources
    Then 100 μl of the mix was transferred into 96-well microplates with monolayer Vero cells.
    Vero
    suggested: CLS Cat# 605372/p622_VERO, RRID:CVCL_0059)
    Software and Algorithms
    SentencesResources
    Antibody tests: We used three different antibody tests throughout the study: 1) chemiluminescent microparticle immunoassay Abbott Archi-tect SARS-CoV-2 IgG on the Abbott ARCHITECT® i2000sr platform (Abbott Laboratories, Chicago, USA), detecting immunoglobulin class G (IgG) antibodies to the nucleocapsid protein of SARS-CoV-2 with the signal/cut-off (S/CO) ratio of 1.4 for positivity (CMIA Abbot; www.fda.gov/media/137383/download).;
    Abbott
    suggested: (Abbott, RRID:SCR_010477)
    Abbott Laboratories
    suggested: None

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This study has several limitations. PCR test results that were chosen as a golden standard, were carried out in different laboratories. Although official test certificates that are registered in the national database were provided, false-positive results cannot be ruled out [28]. But as mentioned above, PCR false-positivity is likely to underestimate antibody test sensitivity in this study. Neutralization test results should be considered with caution as well [29]. Such tests can be considered as a surrogate marker of protection from reinfection, but this association needs to be explored in population-based epidemiological studies [15]. In conclusion, this validation study provides a reference that can be used in further seroprevalence reports to correct the results based on test sensitivity and specificity. Choice of the test for longitudinal surveillance is critical for making conclusions about the spread of SARS-CoV-2 and the durability of the immune response. Local tests should be rigorously evaluated for seroprevalence studies because the benefits of using properly validated tests are not only financial or related to matters of convenience. They may provide more accurate and unbiased assessments for the course of the pandemic.

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04406038Active, not recruitingStudy of the Spread of COVID-19 in Saint Petersburg, Russia
    ISRCTN11060415NANA

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.04.05.21254712 on medRxiv