Performance and Operational Evaluation of the Access Bio CareStart Rapid Antigen Test in a High-Throughput Drive-Through Community Testing Site in Massachusetts

  1. Nira R Pollock
  2. Kristine Tran
  3. Jesica R Jacobs
  4. Amber E Cranston
  5. Sita Smith
  6. Claire Y O’Kane
  7. Tyler J Roady
  8. Anne Moran
  9. Alison Scarry
  10. Melissa Carroll
  11. Leila Volinsky
  12. Gloria Perez
  13. Pinal Patel
  14. Stacey Gabriel
  15. Niall J Lennon
  16. Lawrence C Madoff
  17. Catherine Brown
  18. Sandra C Smole

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Background

To facilitate deployment of point-of-care testing for severe acute respiratory syndrome coronavirus 2, we evaluated the Access Bio CareStart COVID-19 Antigen test in a high-throughput, drive-through, free community testing site using anterior nasal (AN) swab reverse-transcription polymerase chain reaction (RT-PCR) for clinical testing.

Methods

Consenting symptomatic and asymptomatic children (≤18 years) and adults received dual AN swabs. CareStart testing was performed with temperature/humidity monitoring. All tests had 2 independent reads to assess interoperator agreement. Patients with positive CareStart results were called and instructed to isolate pending RT-PCR results. The paired RT-PCR result was the reference for sensitivity and specificity calculations.

Results

Of 1603 participants, 1245 adults and 253 children had paired RT-PCR/CareStart results and complete symptom data. Eighty-three percent of adults and 87% of children were asymptomatic. CareStart sensitivity/specificity were 84.8% (95% confidence interval [CI], 71.1–93.7)/97.2% (95% CI, 92.0–99.4) and 85.7% (95% CI, 42.1–99.6)/89.5% (95% CI, 66.9–98.7) in adults and children, respectively, within 5 days of symptoms. Sensitivity/specificity were 50.0% (95% CI, 41.0–59.0)/99.1% (95% CI, 98.3–99.6) in asymptomatic adults and 51.4% (95% CI, 34.4–68.1)/97.8% (95% CI, 94.5–99.4) in asymptomatic children. Sensitivity in all 234 RT-PCR-positive people was 96.3% with cycle threshold (Ct) ≤25, 79.6% with Ct ≤30, and 61.4% with Ct ≤35. All 21 false-positive CareStart tests had faint but normal bands. Interoperator agreement was 99.5%. Operational challenges included identification of faint test bands and inconsistent swab elution volumes.

Conclusions

CareStart had high sensitivity in people with Ct ≤25 and moderate sensitivity in symptomatic people overall. Specificity was unexpectedly lower in symptomatic versus asymptomatic people. Excellent interoperator agreement was observed, but operational challenges indicate that operator training is warranted.

Article activity feed

  1. Version published to 10.1093/ofid/ofab243
  2. ScreenIT

    SciScore for 10.1101/2021.03.07.21253101: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board StatementConsent: Verbal consent for dual AN swabbing was obtained from adults and guardians of minors (with verbal assent for ages 7-17).
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Analyses utilized Microsoft Excel and GraphPad Prism.
    Microsoft Excel
    suggested: (Microsoft Excel, RRID:SCR_016137)
    GraphPad Prism
    suggested: (GraphPad Prism, RRID:SCR_002798)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study had some limitations. We recognize that the comparator in our study was RT-PCR performed on an AN swab, as opposed to an NP swab, which is still considered the reference method by the FDA [18]. This dual AN swab study design was also used for our recent BinaxNOW study [6]. Although AN swabs have had lower sensitivity than NP swabs in some studies, the sensitivity is highly dependent on the sampling technique and assay used [19]. The dry AN swab sampling method used in this study has been shown to have similar sensitivity to paired NP swabs in transport media [13]. We also note that a recent comparison study demonstrated that Ag RDT performance with nasal mid-turbinate swabs was similar to Ag RDT performance with NP swabs [11]. The time interval between sample collection and test initiation in this study is discussed above. Finally, we recognize that our symptomatic pediatric cohort was relatively small and thus the confidence intervals on all performance estimates relatively wide. In summary, the Access Bio CareStart Ag RDT had high sensitivity in individuals with high viral burden (Ct ≤25) and moderate sensitivity in symptomatic individuals overall. Observed specificity was lower than estimates in the manufacturer IFU, slightly lower than some other visually-read Ag RDT products on the market, and unexpectedly lower in symptomatic versus asymptomatic individuals, warranting additional study. Excellent inter-operator agreement was observed, but operational challenge...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a protocol registration statement.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.03.07.21253101v1 on medRxiv