Evaluation of SARS-CoV-2 antibody point of care devices in the laboratory and clinical setting
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
SARS-CoV-2 antibody tests have been marketed to diagnose previous SARS-CoV-2 infection and as a test of immune status. There is a lack of evidence on the performance and clinical utility of these tests. We aimed to carry out an evaluation of 14 point of care (POC) SARS-CoV-2 antibody tests. Serum from participants with previous RT-PCR (real-time polymerase chain reaction) confirmed SARS-CoV-2 infection and pre-pandemic serum controls were used to determine specificity and sensitivity of each POC device. Changes in sensitivity with increasing time from infection were determined on a cohort of study participants. Corresponding neutralising antibody status was measured to establish whether the detection of antibodies by the POC device correlated with immune status. Paired capillary and serum samples were collected to ascertain whether POC devices performed comparably on capillary samples. Sensitivity and specificity varied between the POC devices and in general did not meet the manufacturers’ reported performance characteristics, which signifies the importance of independent evaluation of these tests. The sensitivity peaked at ≥20 days following onset of symptoms, however sensitivity of 3 of the POC devices evaluated at extended time points showed that sensitivity declined with time. This was particularly marked at >140 days post infection. This is relevant if the tests are to be used for sero-prevalence studies. Neutralising antibody data showed that positive antibody results on POC devices did not necessarily confer high neutralising antibody titres, and that these POC devices cannot be used to determine immune status to the SARS-CoV-2 virus. Comparison of paired serum and capillary results showed that there was a decline in sensitivity using capillary blood. This has implications in the utility of the tests as they are designed to be used on capillary blood by the general population.
Article activity feed
-
-
SciScore for 10.1101/2021.12.16.21267703: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: Outpatient COVID-19 convalescent patient samples were obtained through the NRS BioResource (Ref: SR1407) and the COVID-19 Antibody Test Evaluation (CATE) study, with ethical approval from London-Brent Research Ethics Committee (REC ref: 20/HRA/3764 IRAS: 286538 Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Table 2: Resources
Antibodies Sentences Resources Antibody results were visually read and recorded as positive, weak positive, or negative for IgG and IgM. IgMsuggested: NoneSoftware and Algorithms Sentences Resources All data were analysed using PRISM version 9 and SPSS, and a p value<0.05 was considered significant. PRISMsu…SciScore for 10.1101/2021.12.16.21267703: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: Outpatient COVID-19 convalescent patient samples were obtained through the NRS BioResource (Ref: SR1407) and the COVID-19 Antibody Test Evaluation (CATE) study, with ethical approval from London-Brent Research Ethics Committee (REC ref: 20/HRA/3764 IRAS: 286538 Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Table 2: Resources
Antibodies Sentences Resources Antibody results were visually read and recorded as positive, weak positive, or negative for IgG and IgM. IgMsuggested: NoneSoftware and Algorithms Sentences Resources All data were analysed using PRISM version 9 and SPSS, and a p value<0.05 was considered significant. PRISMsuggested: (PRISM, RRID:SCR_005375)SPSSsuggested: (SPSS, RRID:SCR_002865)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:However, the study does have limitations. Due to sample availability constraints and the large number of POCTs evaluated not all could be evaluated on the same panel of positive and negative samples. In particular, the numbers of samples where paired serum and capillary sample results were available to assess concordance was limited for the majority of the POCT examined. For one of the test kits ease of use was assessed but as many of the study participants providing capillary samples in this study were healthcare workers the data obtained through this may not be representative of the general population. In summary, our results highlight a wide variation in performance of SARS-CoV-2 antibody test kits and illustrates the importance of evaluating multiple different aspects of test performance including checking for batch to batch variation, changes in sensitivity as time from infection increases, correlation with neutralising antibodies and performance on capillary samples. Thorough evaluation of all these aspects is essential prior to considering the utilisation of these tests for antibody or ‘immunity’ passports and identification of hospitalised patients who would benefit from monoclonal antibody treatment.
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
