Validation and performance comparison of three SARS‐CoV‐2 antibody assays

  1. Kimberly J. Paiva
  2. Ricky D. Grisson
  3. Philip A. Chan
  4. Richard C. Huard
  5. Angela M. Caliendo
  6. John R. Lonks
  7. Ewa King
  8. Eric W. Tang
  9. Diane L. Pytel‐Parenteau
  10. Ga H. Nam
  11. Evgeny Yakirevich
  12. Shaolei Lu

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Serology testing of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) is increasingly being used during the current pandemic of coronavirus disease 2019 (COVID‐19), although its clinical and epidemiologic utilities are still debatable. Characterizing these assays provides scientific basis to best use them. The current study assessed one chemiluminescent assay (Abbott COVID‐2 IgG) and two lateral flow assays (STANDARD Q [SQ] IgM/IgG Duo and Wondfo total antibody test) using 113 blood samples from 71 PCR‐confirmed COVID‐19 hospitalized patients, 119 samples with potential cross‐reactions, and 1068 negative controls including 942 pre‐pandemic samples. SARS‐CoV‐2 IgM antibodies became detectable 3‐4 days post‐symptom onset using SQ IgM test and IgG antibodies were first detected 5‐6 days post‐onset using SQ IgG. Abbott IgG and Wondfo Total were able to detect antibodies 7 to 8 days post‐onset. After 14 days post‐symptom onset, the SQ IgG, Abbott IgG and Wondfo Total tests were able to detect antibodies from 100% of the PCR‐confirmed patients in this series; 87.5% sensitivity for SQ IgM. Overall agreement was 88.5% between SQ IgM/IgG and Wondfo Total and 94.6% between SQ IgG and Abbott IgG. No cross‐reaction due to recent sera with three of the endemic coronaviruses was observed. Viral hepatitis and autoimmune samples were the main source of limited cross‐reactions. The specificities were 100% for SQ IgG and Wondfo Total, 99.62% for Abbott IgG, and 98.87% for SQ IgM. These findings demonstrated high sensitivity and specificity of appropriately validated SARS‐CoV‐2 serologic assays with implications for clinical use and epidemiological seroprevalence studies.

Article activity feed

  1. Version published to 10.1002/jmv.26341
  2. ScreenIT

    SciScore for 10.1101/2020.05.29.124776: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: The study was approved by the Institutional Review Board (IRB) of Lifespan Health System (including Rhode Island Hospital and The Miriam Hospital) to ensure the study met the ethical requirements.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    119 samples that were positive for antibodies against viruses and other pathogens were used to test cross-reaction of the assays (Table3).
    Table3
    suggested: None
    Additional samples were collected consisting of interference antibodies such as Rheumatoid factor (RF), anti-double strand DNA (ds-DNA), anti-nuclear antibody (ANA) and paraprotein IgM and IgG (Table 3).
    RF), anti-double strand DNA
    suggested: None
    anti-double
    suggested: None
    anti-nuclear
    suggested: None
    Lateral flow assays: SARS-CoV-2 Total Antibody Test (Wondfo, Guangzhou, China) and STANDARD Q COVID-19 IgM/IgG Duo Test kits (SD Biosensor, Gyeonggi-do, Korea) were purchased from the manufacturers and the assays were performed following the manufacturer’s protocols (8, 9).
    Test (Wondfo, Guangzhou, China)
    suggested: None
    Gyeonggi-do, Korea
    suggested: None
    Software and Algorithms
    SentencesResources
    To obtain more precise specificities for SQ IgM, SQ IgG and Abbott IgG, 1063 serum or plasma samples were collected before the pandemic started in the United States (January 2020), including 500 samples originally for reference range determination of a troponin assay, 371 prenatal samples for reference range determination of quadruple tests, 50 pre-pandemic samples from transfusion service and 21 pre-pandemic plasma segments from the Rhode Island Blood Center.
    Abbott
    suggested: (Abbott, RRID:SCR_010477)
    The assays were performed on an Abbott Architect i1000 analyzer following the manufacturer’s protocol.
    Abbott Architect
    suggested: (Abbott ARCHITECT i1000sr System, RRID:SCR_019328)
    Data analysis: The data collected were analyzed on a statistical package, JMP Pro 14.0 (SAS Institute, Cary, NC).
    SAS Institute
    suggested: (Statistical Analysis System, RRID:SCR_008567)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitation of the study: The main limitation of the study is that the samples from COVID-19 patients were collected from an inpatient population. This group of patients were generally overweight or obese (77%) with high prevalence of diabetes (40%). They tended to have a high D-Dimer levels and marked lymphocytopenia. The clinical symptoms tended to be severe with more being intubated and poor clinical outcomes. The antibody response in this population has been shown to be robust (17). In outpatient population, asymptomatic infected individuals have been reported only with ∼ 10% (28/276) seropositive rates (20). Moreover, asymptomatic and pauci-symptomatic patients could have no detectable antibody response 4 weeks after the diagnosis (21). Another limitation is the limited number of non-COVID positive samples collected at least 2 weeks post symptom onset. More work is needed to assess these tests in this patient population. Another important question in COVID-19 serology is how long the antibody response will persist. The three samples collected over 30 days post symptom onset had Abbott S/CO reading of 7.58 (31 days), 6.37 (31 days) and 2.43 (35 days). The last one was from a patient with end stage renal disease which is known for its attenuated immune response. The other two were among the most robust immune responses in this cohort (both over 90% quantile of S/CO readings). Obviously, follow-up testing of these patients’ antibody S/CO levels will help to answer the questi...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • No conflict of interest statement was detected. If there are no conflicts, we encourage authors to explicit state so.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.05.29.124776v1 on bioRxiv