Clinical Evaluation of a COVID-19 Antibody Lateral Flow Assay using Point of Care Samples

  1. Won Lee
  2. Steven Straube
  3. Ryan Sincic
  4. Jeanne A. Noble
  5. Juan Carlos Montoy
  6. Aaron E. Kornblith
  7. Arun Prakash
  8. Ralph Wang
  9. Roland J. Bainton
  10. Philip A. Kurien

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Introduction

The ongoing SARS-CoV-2 pandemic has spurred the development of numerous point of care (PoC) immunoassays. Assessments of performance of available kits are necessary to determine their clinical utility. Previous studies have mostly performed these assessments in a laboratory setting, which raises concerns of translating findings for PoC use. The aim of this study was to assess the performance of a lateral flow immunoassay for the detection of SARS-CoV-2 antibodies using samples collected at PoC.

Method

One lateral flow immunoassay (Humasis ® COVID-19 IgG/IgM) was tested. In total, 50 PCR RT-PCR positive and 52 RT-PCR negative samples were collected at PoC. Fifty serum specimens from Dec 2018 to Feb 2019 were used as controls for specificity. Serum samples collected between Dec 2019 to Feb 2020 were used as additional comparators. Clinical data including symptom onset date was collected from patient history and the medical record.

Results

The overall sensitivity for the kit was 74% (95% CI: 59.7% -85.4%). The sensitivity for IgM and IgG detection >14 days after date of onset was 88% (95% CI: 68.8% -97.5%) and 84% (95% CI: 63.9% – 95.5%), with a negative predictive value (NPV) of 94% for IgM (95% CI: 83.5% - 98.8%) and 93% for IgG (95% CI: 81.8% - 97.9%). The overall specificity was 94% (95% CI: 83.5% - 98.8%). The Immunoglobulin specific specificity was 94% for IgM (95% CI: 83.5% - 98.8%) and 98% for IgG (95% CI: 89.4% - 100.0%), with a positive predictive value (PPV) of 88% for IgM (95% CI: 68.8% - 97.5%) and 95% for IgG (95% CI: 77.2% - 99.9%) respectively for samples collected from patients >14 days after date of onset. Specimen collected during early phase of COVID-19 pandemic (Dec 2019 to Feb 2020) showed 11.8% antibody positivity, and 11.3% of PCR-negative patients demonstrated antibody positivity.

Discussion

Humasis ® COVID-19 IgG/IgM LFA demonstrates greater than 90% PPV and NPV for samples collected 14 days after the onset of symptoms using samples collected at PoC. While not practical for the diagnosis of acute infection, the use of the lateral flow assays with high specificity may have utility for determining seroprevalence or seroconversion in longitudinal studies.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.12.02.20242750: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementIRB: Ethical Approvals: This study was approved by the institutional review board at the University of California, San Francisco (UCSF) and Zuckerberg San Francisco General Hospital (ZSFG).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    For analysis, two clinician assessors blinded to the specimen’s PCR status read the result on a cartridge and assigned a binary score (0 for negative, 1 for positive) for immunoglobulin M (IgM) and immunoglobulin G (IgG) antibodies; a third adjudicated disagreements by assigning a binary score.
    immunoglobulin G (IgG
    suggested: None
    Software and Algorithms
    SentencesResources
    Data was aggregated and processed in Python (Pycharm®, Prague, Czech Republic), and subsequent analyses were conducted using STATA 15.1 (College Station, TX).
    Python
    suggested: (IPython, RRID:SCR_001658)
    STATA
    suggested: (Stata, RRID:SCR_012763)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitations: A few study limitations should be noted. Notably, our samples were collected from the San Francisco Bay Area. Different strains of SARS-CoV-2 exist around the world [19], and whether our result will be reproducible for other strains is unknown [20]. A second limitation is that the samples used to evaluate the specificity as true-negatives were from bio-banked plasma samples and were carried out in a controlled laboratory setting. Also, a majority of our PCR positive patients were admitted to the hospital, which limits generalizability. Lastly, we have compared the LFA test results to those of RT-PCR. While PCR test remains the molecular gold standard for COVID-19 diagnosis, PCR tests, themselves, are subject to inherent diagnostic uncertainties [21], and comparing the presence of viral RNA in oral or nasopharyngeal secretions to serum antibodies has drawbacks. Therefore, our findings may not truly reflect false positives and false negatives, which would be better further validated with other molecular antibody tests, such as serum IgG ELISAs. Ng et al. have identified SARS-CoV-2 S-reactive IgG antibodies from May 2019 samples, representing preexisting humoral immunity from exposure to other coronavirus strains [22]. ELISA assays may have helped to better differentiate if false positives were due to existence of the antibodies. Similarly, false negative data points may be observed due to the failure or a delay in mounting antibody responses. For purposes of this s...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.12.02.20242750v1 on medRxiv