Validation and Performance Comparison of Three SARS-CoV-2 Antibody Assays

  1. Kimberly J Paiva
  2. Ricky D Grisson
  3. Philip A Chan
  4. John R. Lonks
  5. Ewa King
  6. Richard C Huard
  7. Diane L Pytel-Parenteau
  8. Ga Hie Nam
  9. Evgeny Yakirevich
  10. Shaolei Lu

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Serology testing of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is increasingly being used during the current pandemic of Coronavirus Disease 2019 (COVID-19). The clinical and epidemiologic utilities of antibody-based SARS-CoV-2 testing are under debate. Characterizing these assays helps to understand the disease and provides scientific basis for deciding how to best use these assays. The study assessed one chemiluminescent assay (Abbott COVID-2 IgG) and two lateral flow assays (STANDARD Q [SQ] IgM/IgG Duo and Wondfo Total Antibody Test). Validation included 113 blood samples from 71 PCR-confirmed COVID-19 patients and 1182 samples from negative controls with potential interferences/cross-reactions, including 1063 pre-pandemic samples. IgM antibodies against SARS-CoV-2 were detected as early as post-symptom onset days 3-4. IgG antibodies were first detected post-onset days 5-6 by SQ assays. The detection rates increased gradually, and SQ IgG, Abbott IgG and Wondfo Total detected antibodies from all the PCR-confirmed patients 14 days after symptom onset. Overall agreements between SQ IgM/IgG and Wondfo Total was 88.5% and between SQ IgG and Abbott IgG was 94.6% (Kappa = 0.75, 0.89). No cross-reaction with other endemic coronavirus infections were identified. Viral hepatitis and autoimmune samples were the main cross-reactions observed. However, the interferences/cross-reactions were low. The specificities were 100% for SQ IgG and Wondfo Total and 99.62% for Abbott IgG and 98.87% for SQ IgM. These findings demonstrate high sensitivity and specificity of appropriately validated antibody-based SARS-CoV-2 assays with implications for clinical use and epidemiological seroprevalence studies.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.05.29.124776: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: The study was approved by the Institutional Review Board (IRB) of Lifespan Health System (including Rhode Island Hospital and The Miriam Hospital) to ensure the study met the ethical requirements.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    119 samples that were positive for antibodies against viruses and other pathogens were used to test cross-reaction of the assays (Table3).
    Table3
    suggested: None
    Additional samples were collected consisting of interference antibodies such as Rheumatoid factor (RF), anti-double strand DNA (ds-DNA), anti-nuclear antibody (ANA) and paraprotein IgM and IgG (Table 3).
    RF), anti-double strand DNA
    suggested: None
    anti-double
    suggested: None
    anti-nuclear
    suggested: None
    Lateral flow assays: SARS-CoV-2 Total Antibody Test (Wondfo, Guangzhou, China) and STANDARD Q COVID-19 IgM/IgG Duo Test kits (SD Biosensor, Gyeonggi-do, Korea) were purchased from the manufacturers and the assays were performed following the manufacturer’s protocols (8, 9).
    Test (Wondfo, Guangzhou, China)
    suggested: None
    Gyeonggi-do, Korea
    suggested: None
    Software and Algorithms
    SentencesResources
    To obtain more precise specificities for SQ IgM, SQ IgG and Abbott IgG, 1063 serum or plasma samples were collected before the pandemic started in the United States (January 2020), including 500 samples originally for reference range determination of a troponin assay, 371 prenatal samples for reference range determination of quadruple tests, 50 pre-pandemic samples from transfusion service and 21 pre-pandemic plasma segments from the Rhode Island Blood Center.
    Abbott
    suggested: (Abbott, RRID:SCR_010477)
    The assays were performed on an Abbott Architect i1000 analyzer following the manufacturer’s protocol.
    Abbott Architect
    suggested: (Abbott ARCHITECT i1000sr System, RRID:SCR_019328)
    Data analysis: The data collected were analyzed on a statistical package, JMP Pro 14.0 (SAS Institute, Cary, NC).
    SAS Institute
    suggested: (Statistical Analysis System, RRID:SCR_008567)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitation of the study: The main limitation of the study is that the samples from COVID-19 patients were collected from an inpatient population. This group of patients were generally overweight or obese (77%) with high prevalence of diabetes (40%). They tended to have a high D-Dimer levels and marked lymphocytopenia. The clinical symptoms tended to be severe with more being intubated and poor clinical outcomes. The antibody response in this population has been shown to be robust (17). In outpatient population, asymptomatic infected individuals have been reported only with ∼ 10% (28/276) seropositive rates (20). Moreover, asymptomatic and pauci-symptomatic patients could have no detectable antibody response 4 weeks after the diagnosis (21). Another limitation is the limited number of non-COVID positive samples collected at least 2 weeks post symptom onset. More work is needed to assess these tests in this patient population. Another important question in COVID-19 serology is how long the antibody response will persist. The three samples collected over 30 days post symptom onset had Abbott S/CO reading of 7.58 (31 days), 6.37 (31 days) and 2.43 (35 days). The last one was from a patient with end stage renal disease which is known for its attenuated immune response. The other two were among the most robust immune responses in this cohort (both over 90% quantile of S/CO readings). Obviously, follow-up testing of these patients’ antibody S/CO levels will help to answer the questi...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • No conflict of interest statement was detected. If there are no conflicts, we encourage authors to explicit state so.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.05.29.124776 on bioRxiv