Performance of Existing and Novel Surveillance Case Definitions for COVID-19 in the Community

  1. Hannah E. Reses
  2. Mark Fajans
  3. Scott H. Lee
  4. Charles M. Heilig
  5. Victoria T. Chu
  6. Natalie J. Thornburg
  7. Kim Christensen
  8. Sanjib Bhattacharyya
  9. Alicia Fry
  10. Aron J. Hall
  11. U.S. COVID-19 Household Investigation Team
  12. Jacqueline E. Tate
  13. Hannah L. Kirking
  14. Scott A. Nabity

Reviewed by

Read the full article

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Background

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes coronavirus disease 2019 (COVID-19), presents with a broad range of symptoms. Existing COVID-19 case definitions were developed from early reports of severely ill, primarily hospitalized, patients. Symptom-based case definitions that guide public health surveillance and individual patient management in the community must be optimized for COVID-19 pandemic control.

Methods

We collected daily symptom diaries and performed RT-PCR on respiratory specimens over a 14-day period in 185 community members exposed to a household contact with COVID-19 in the Milwaukee, Wisconsin and Salt Lake City, Utah metropolitan areas. We interpreted the discriminatory performance (sensitivity, specificity, predictive values, F⍰1 score, Youden’s index, and prevalence estimation) of individual symptoms and common case definitions according to two principal surveillance applications (i.e., individual screening and case counting). We also constructed novel case definitions using an exhaustive search with over 73 million symptom combinations and calculated bias-corrected and accelerated bootstrap confidence intervals stratified by children versus adults.

Findings

Common COVID-19 case definitions generally showed high sensitivity (86⍰96%) but low positive predictive value (PPV) (36⍰49%; F⍰1 score 52⍰63) in this community cohort. The top performing novel symptom combinations included taste or smell dysfunction. They also improved the balance of sensitivity and PPV (F⍰1 score 78⍰80) and reduced the number of false positive symptom screens. Performance indicators were generally lower for children (<18 years of age).

Interpretation

Existing COVID-19 case definitions appropriately screened in community members with COVID-19. However, they led to many false positive symptom screens and poorly estimated community prevalence. Absent unlimited, timely testing capacity, more accurate case definitions may help focus public health resources. Novel symptom combinations incorporating taste or smell dysfunction as a primary component better balanced sensitivity and specificity. Case definitions tailored specifically for children versus adults should be further explored.

Funding

This research was wholly supported by the U.S. Centers for Disease Control and Prevention.

Disclaimer

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the U.S. Centers for Disease Control and Prevention/the Agency for Toxic Substances and Disease Registry.

Research in Context

Evidence before this study

Coronavirus disease 2019 (COVID-19) incidence has accelerated globally over the last several months. As the full spectrum of clinical presentations has come into clearer focus, symptom-based clinical screening and case surveillance has also evolved. Preliminary understanding of the clinical manifestation of COVID-19 was driven primarily by descriptions of hospitalized patients, as early testing algorithms prioritized more severely ill persons with classic lower respiratory symptoms and fever. Since then, more data from ambulatory settings have emerged. We searched PubMed from 1 December 2019 to 21 August 2020 for studies that assessed the diagnostic performance of case surveillance definitions for COVID-19. We found no studies examining the discriminatory performance of case surveillance definitions among contacts with mild to moderate symptoms with documented exposure to persons with COVID-19. Nonetheless, we found nine highly relevant studies: seven original reports and two review articles. Five original studies evaluated individual, self-reported symptoms (two among healthcare workers in the United States, one among healthcare workers in the Netherlands, and one online survey for the general public in Somalia) and concluded that using dysfunction of taste or smell for routine COVID-19 screening likely had utility. The fifth study had a similar conclusion based on self-reported symptoms and laboratory results collected via smartphone from the general public in the United States and the United Kingdom. Another original study modeled the substantial effect that multiple revisions to the COVID-19 case definition had on the reported disease burden in the Chinese population. Lastly, an original study illustrated the shift in discriminatory performance of established influenza surveillance case definitions for influenza between adults and children. Age-specific differences in case definition performance may also apply to COVID-19. Two articles reviewed predictive algorithms to define outpatient COVID-19 illness and risk of hospitalization. The reviewed studies were limited in that they were either restricted to individual signs or symptoms, or they incorporated blood tests or imaging that required in-person access to medical care.

Added value of this study

The discriminatory performance of case surveillance definitions for COVID-19 is important for implementing effective epidemic mitigation strategies. Our study illustrates the performance of case definitions in community members with household exposure to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) based solely on symptom profiles. Prior work overrepresented healthcare workers or otherwise studied non-representative populations, and they did not examine across the age spectrum. Our study also provides a novel framework for refining definitions. Using 15 symptoms associated with COVID-19 for all contacts regardless of disease status, we systematically evaluated the discriminatory performance of individual symptoms and previously defined case surveillance definitions across ages and according to two core surveillance applications: 1) screening non-hospitalized individuals to prioritize public health interventions, and 2) estimating the number of non-hospitalized persons with COVID-19 (i.e., community-based syndromic surveillance). We also constructed novel symptom combinations that effectively performed both functions and improved upon widely used case surveillance definitions that may help to target interventions in the absence of unlimited laboratory diagnostic capacity. Our analyses highlight the importance of ongoing re-evaluation of symptom-based surveillance definitions to suit the intended purpose and population under surveillance. Based on our results, which were derived from household members of all ages, case surveillance definition performance may improve if developed separately for adults and children.

Implications of all the available evidence

Case definitions for COVID-19 should be tailored to maximize the discriminatory performance dependent upon its intended use. Existing COVID-19 case definitions screened in most community members with COVID-19, but also yielded a high number of false positive results. When unlimited, timely diagnostic testing is not available symptom combinations with improved accuracy (i.e., more balanced sensitivity and specificity) may help focus resources, such as recommending self-isolation among community contacts.

Article activity feed

  1. ScreenIT

    SciScore for 10.1101/2020.10.02.20195479: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    De-identified data and analytic scripts in R and Python are publicly available through a GitHub repository: https://github.com/scotthlee/covid-casedefs.
    Python
    suggested: (IPython, RRID:SCR_001658)

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  2. Version published to 10.1101/2020.10.02.20195479 on medRxiv