Evaluation of the specificity and accuracy of SARS-CoV-2 rapid antigen self-tests compared to RT-PCR from 1015 asymptomatic volunteers

  1. Thomas Iftner
  2. Angelika Iftner
  3. Diana Pohle
  4. Peter Martus

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Objective

Evaluation of the specificity and accuracy of four CE-approved SARS-CoV-2 antigen rapid self-tests (AG-ST) Anbio, Clungene, Hotgene and Mexacare.

Method

1015 asymptomatic volunteers were screened for SARS-CoV-2 by means of an oropharyngeal swab taken by qualified personnel and subsequent RT-PCR testing. Each participant additionally performed nasal self-swabs for two of the four rapid antigen tests at the same day according to the manufacturers’ instructions. Study participants transmitted a photo and own interpretation of their test results to the study center. The results of the two self-tests provided by the participants were correlated with the results of the SARS-CoV-2 RT-PCR and independently assessed and evaluated by the study center.

Results

None of the volunteers tested positive upon RT-PCR, whereas 13 AG-ST showed a false positive test result (0.7 %). The highest false positivity rate was found for the Clungene test (2.1 % compared to 0.2 % for the other tests), while the highest test failure rate (invalid) was found for the Mexacare test (3.7%). The Anbio and Hotgene tests produced the fewest false positive results when evaluated by the participants and also showed the best agreement among themselves.

Conclusion

SARS-CoV-2 Antigen rapid self-tests with higher false positive test rates, such as the Clungene test, or with high rates of invalid test results, such as the Mexacare test, are less suitable for screening purposes of asymptomatic study participants especially in low-prevalence settings. False positive or inadequate test results increase the burden on certified test laboratories due to verification PCR tests and cause a substantial economic loss due to unnecessary quarantine measurements and cause psychological stress in the affected study participants. In addition to earlier defined requirements for sensitivity for SARS-CoV-2 detection, a lower acceptance boundary for the false positivity rate of < 0.3% should be demanded.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2022.02.11.22270873: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    RandomizationEach participant additionally performed nasal self-swabs without supervision for two of the four randomly selected rapid antigen tests on the day of oropharyngeal swab collection according to the manufacturers’ instructions (the order was determined by the participant) and sent a photograph of the test results by means of a bar-coded form without participant’s name via e-mail to an e-mail address specifically set up for the study.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Statistical analysis was performed by using SPSS (program release 26).
    SPSS
    suggested: (SPSS, RRID:SCR_002865)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    A potential limitation of the study might be a relatively high number of medical staff, e.g. participants likely were better trained to use AG-ST as the rest of the general population. Therefore, handling or interpretation errors could be underestimated in this cohort. As additional criteria might affect the performance of AG-ST it would have been useful to allow the user to describe in a free text field of the form sheet used to transfer the test results to the study center the possible source of interpretation errors, e.g. by lack of readability of the test, missing components in the kits etc. In summary, our data show that the Clungene and Mexacare tests are probably less suitable for usage as antigen self-tests due to their high rate of false positive results and high number of unclear/invalid results, respectively. Tests with a high false-positivity rate should especially be avoided to be used in a population with a low prevalence of SARS-CoV-2.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2022.02.11.22270873v1 on medRxiv