Accuracy and usability of saliva and nasal rapid antigen self-testing for detection of SARS-CoV-2 infection in the general population: a head-to-head comparison
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Background
SARS-CoV-2 self-tests may lower the threshold of testing and produce a result quickly. This could support the early detection of infectious cases and reduce further community transmission. However, the diagnostic accuracy of (unsupervised) self-testing with rapid antigen diagnostic tests (Ag-RDTs) is mostly unknown. We therefore conducted a large-scale head-to-head comparison of the diagnostic accuracy of a self-performed SARS-CoV-2 saliva and nasal Ag-RDT, each compared to a molecular reference test, in the general population in the Netherlands.
Methods
In this cross-sectional study we consecutively included individuals aged 16 years and older presenting for SARS-CoV-2 testing at three Dutch public health service test sites irrespective of their indication for testing, vaccination status, and symptomatology. Participants were sampled for molecular testing at the test site and received two self-tests (the Hangzhou AllTest saliva self-test and the SD Biosensor nasal self-test by Roche Diagnostics) to perform at home within a few hours without knowledge of their molecular test result. Information on presence and type of symptoms, user experiences, and results of both self-tests were collected via an online questionnaire. For each self-test, sensitivity, specificity, positive and negative predictive values were determined with molecular testing as reference standard.
Findings
The SARS-CoV-2 molecular reference test positivity rate was 6.5% in the 2,819 participants. Overall sensitivities with 95% confidence intervals were 46.7% (85/182; 39.3%-54.2%) for the saliva Ag-RDT, and 68.9% (124/180; 61.6%-75.6%) for the nasal Ag-RDT. With a viral load cut-off (≥5.2 log10 SARS-CoV-2 E-gene copies/mL) as a proxy of infectiousness, sensitivities increased to 54.9% (78/142; 46.4%-63.3%) for the saliva Ag-RDT and 83.9% (120/143; 76.9%-89.5%) for the nasal Ag-RDT.
For the nasal Ag-RDT, sensitivities were 78.5% [71.1%-84.8%] and 22.6% [9.6%-41.1%] in those with and without symptoms at the time of sampling, which increased to 90.4% (113/125; 83.8%-94.9%) and 38.9% (7/18; 17.3%-64.3%) after applying the viral load cut-off. In those with and without prior confirmed SARS-CoV-2, sensitivities were 36.8% [19/372; 16.3%-61.6%] and 72.7% [161/2437; 65.1%-79.4%] for the nasal Ag-RDT, which increased to 100% (7/7; 59.0%-100%) and 83.1% (113/126; 75.7%-89.0%) after applying the viral load cut-off.
The diagnostic accuracy of the nasal Ag-RDT did not differ by COVID-19 vaccination status, sex, and age. Specificities were >99%, positive predictive values >70% and negative predictive values >95%, for the saliva Ag-RDT, and >99%, >90%, and >95% for the nasal Ag-RDT, respectively, in most analyses.
Interpreting the results was considered (very) easy for both self-tests.
Interpretation
The Hangzhou AllTest self-performed saliva Ag-RDT is not reliable for SARS-CoV-2 infection detection overall nor in the studied subgroups. The SD Biosensor self-performed nasal Ag-RDT had high sensitivity in individuals with symptoms and in those without a prior SARS-CoV-2 infection. The overall accuracy in individuals with symptoms was comparable to that found in previous studies with professional sampling for this Ag-RDT. The extremely low sensitivity of the nasal Ag-RDT in asymptomatic individuals and in individuals who had had a prior SARS-CoV-2 infection is an important finding and warrants further investigation.
Funding
Dutch Ministry of Health, Welfare, and Sport.
Article activity feed
-
SciScore for 10.1101/2021.12.08.21267452: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics Consent: Individuals were considered eligible if they were aged 16 years or older, and willing and able to sign a digital informed consent in Dutch. Sex as a biological variable Secondary outcomes were diagnostic accuracies stratified by presence of symptoms at the time of sampling (yes or no), COVID-19 vaccination status (vaccinated with at least one dose yes or no), having had a prior SARS-CoV-2 infection (yes or no), gender (female or male), and age (≥16 to ≤40 or >40 to ≤65 or >65 years). Randomization not detected. Blinding not detected. Power Analysis 13 We therefore based our sample size calculation on an expected sensitivity of 80% for each self-performed Ag-RDT, with a margin of error of … SciScore for 10.1101/2021.12.08.21267452: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics Consent: Individuals were considered eligible if they were aged 16 years or older, and willing and able to sign a digital informed consent in Dutch. Sex as a biological variable Secondary outcomes were diagnostic accuracies stratified by presence of symptoms at the time of sampling (yes or no), COVID-19 vaccination status (vaccinated with at least one dose yes or no), having had a prior SARS-CoV-2 infection (yes or no), gender (female or male), and age (≥16 to ≤40 or >40 to ≤65 or >65 years). Randomization not detected. Blinding not detected. Power Analysis 13 We therefore based our sample size calculation on an expected sensitivity of 80% for each self-performed Ag-RDT, with a margin of error of 7%, type I error of 5% and power of 80%. Table 2: Resources
Software and Algorithms Sentences Resources The study is reported according to the STARD 2015 guidelines: an updated list of essential items for reporting diagnostic accuracy studies. STARDsuggested: NoneResults from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Strengths and limitations of this study: Strengths of this study include the large overall sample size covering multiple test sites nationwide, collection of samples for the reference and two Ag-RDTs in the same individuals within a few hours allowing for a head-to-head comparison of the two self-tests, sampling done by the participants themselves without any supervision conform the real-world context of self-testing, blinding of the index for the reference test result and vice versa, and the use of a proxy for infectiousness. Furthermore, the follow-up information showed that very few infections were missed by the molecular reference tests. Our study also has some limitations. First, the reference standards that we used were molecular tests, but platforms and test kits used differed among the centralized laboratories. However, the diagnostic accuracies of all molecular tests used are similarly high25,26, and we therefore believe that this has not influenced our findings significantly. In addition, Ct values used to calculate viral loads were determined by different yet comparable platforms (supplementary material 3). Second, we used the viral load cut-off above which 95% of people with a positive RT-PCR test result had a positive virus culture as a proxy of infectiousness. Although this cut-off is not fully evidence based3, it is a best estimate based on current knowledge, and less arbitrary than using Ct cut-offs of 25 or 30 as is often done.27,28 In the current study, we r...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
