Testing for coronavirus (SARS-CoV-2) infection in populations with low infection prevalence: the largely ignored problem of false positives and the value of repeat testing

  1. Cathie Sudlow
  2. Peter Diggle
  3. Oliver Warlow
  4. David Seymour
  5. Ben Gordon
  6. Rhos Walker
  7. Charles Warlow

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

Calls are increasing for widespread SARS-CoV-2 infection testing of people from populations with a very low prevalence of infection. We quantified the impact of less than perfect diagnostic test accuracy on populations, and on individuals, in low prevalence settings, focusing on false positives and the role of confirmatory testing.

Methods

We developed a simple, interactive tool to assess the impact of different combinations of test sensitivity, specificity and infection prevalence in a notional population of 100,000. We derived numbers of true positives, true negatives, false positives and false negatives, positive predictive value (PPV – the percentage of test positives that are true positives) and overall test accuracy for three testing strategies: (1) single test for all; (2) add repeat testing in test positives; (3) add further repeat testing in those with discrepant results. We also assessed the impact on test results for individuals having one, two or three tests under these three strategies.

Results

With sensitivity of 80%, infection prevalence of 1 in 2,000, and specificity 99.9% on all tests, PPV in the tested population of 100,000 will be only 29% with one test, increasing to > 99.5% (100% when rounded to the nearest %) with repeat testing in strategies 2 or 3. More realistically, if specificity is 95% for the first and 99.9% for subsequent tests, single test PPV will be only 1%, increasing to 86% with repeat testing in strategy 2, or 79% with strategy 3 (albeit with 6 fewer false negatives than strategy 2). In the whole population, or in particular individuals, PPV increases as infection becomes more common in the population but falls to unacceptably low levels with lower test specificity.

Conclusion

To avoid multiple unnecessary restrictions on whole populations, and in particular individuals, from widespread population testing for SARS-CoV-2, the crucial roles of extremely high test specificity and of confirmatory testing must be fully appreciated and incorporated into policy decisions.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.08.19.20178137: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    NIH rigor criteria are not applicable to paper type.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    We implemented a probabilistic calculation as an interactive tool within a Microsoft Excel spreadsheet workbook (https://www.hdruk.ac.uk/projects/false-positives/).
    Microsoft Excel
    suggested: (Microsoft Excel, RRID:SCR_016137)
    Here we use the tool’s outputs to illustrate a few plausible scenarios that could arise when population prevalence of infection is low, as is currently the case in the UK.
    tool’s
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    However, although simplicity and transparency make our tool attractive, this does result in some limitations. We have assumed that all rounds of testing are independent of each other, whereas in real world situations, there may be characteristics of particular individuals (e.g., swabbing technique for self-administered tests), sample transport systems, or laboratories (e.g. a tendency towards repeated administrative or sample handling errors) that make them inherently more prone to repeatedly inaccurate results. This could be mitigated by ensuring that confirmatory tests are not self-administered, use a different transport mechanism and are conducted in a different laboratory, fully independent of the first, as well as using a different molecular diagnostic test, but this requirement would pose significant logistical challenges. Further, since the proposed testing strategies assume that repeat testing is immediate (and so unaffected by changes in viral detection over the time course of infection), our tool is only applicable for tests with very fast turnaround of results. These are now becoming available,10 but a detailed understanding of their performance, including sensitivity and specificity, will be required to assess their potential use in widespread testing among asymptomatic people. In conclusion, we agree with others that population-wide testing may have an important role to play in determining local, regional and national policy for the population and in providing gu...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.08.19.20178137v1 on medRxiv