Individual and Population Level Uncertainty Interact to Determine the Performance of Outbreak Surveillance Systems
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Outbreak detection frequently relies on imperfect individual-level case diagnosis. Both outbreaks and cases are discrete events that can be misclassified and uncertainty at the case level may impact the performance of outbreak alert and detection systems. Here, we describe how the performance of outbreak detection depends on individual-level diagnostic test characteristics and population-level epidemiology and describe settings where imperfect individual-level tests can achieve consistent performance comparable to “perfect” diagnostic tests.
Methodology
We generated a stochastic SEIR model to simulate daily incidence of measles (i.e., true) and non-measles (i.e., noise) febrile rash illness. We modeled non-measles sources as either independent static (Poisson) noise, or dynamical noise consistent with an independent SEIR process (e.g., rubella). Defining outbreak alerts as the exceedance of a threshold by the 7-day rolling average of observed test positives, we optimized the threshold that maximized outbreak detection accuracy across set of noise structures and magnitudes, diagnostic test accuracy (consistent with either a perfect test, or proposed rapid diagnostic tests), and testing rates.
Conclusions
The optimal threshold for each diagnostic test typically increased monotonically with testing rate. With static noise, outbreak detection with RDT-like and perfect tests achieved accuracies of 90%, with comparable delays to outbreak detection. With dynamical noise, the accuracy of perfect test scenarios was superior to those achieved with RDTs (≈ 90% vs.≈ 80%). Outbreak detection accuracy declined as dynamical noise increased and leads to permanent alert status with RDT-like tests at very high noise. The performance of an outbreak detection system is highly sensitive to the structure and the magnitude of the background noise. Depending on the epidemiological context, outbreak detection using RDTs can perform as well as perfect tests.
Author Summary
To respond to outbreaks of infectious diseases, we first need to detect them. This detection is inherently flawed, in part, due to imperfect diagnostic tests used to indicate whether individuals are positive or negative for a disease. We evaluated the impact of imperfect diagnostic tests for infectious disease on the accuracy and timeliness of outbreak detection in the context of a set of background infections that could be mistaken for the disease of interest, and consequently cause false positive test results. We find that when outbreak detection performance is highly dependent on the structure and magnitude of the background “noise” infections. When the rate of background infections far exceeds that of the target infection, and are dynamical, such that there are large peaks and troughs of “noise infections”, imperfect diagnostic tests are not able to accurately distinguish the “signal” (target infections) from the background “noise”. If the background “noise” infections are either less cyclical in their dynamics, or do not outnumber true infections by a great deal, imperfect diagnostic tests can perform well.
