Improving Real-world Antigen Test Sensitivity Estimation through Target Distribution Balancing
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Sensitivity is a key measure of lateral-flow antigen test (AT) performance, typically compared against qRT-PCR as the gold standard. For COVID-19, diagnostic sensitivity assesses the ability of ATs to detect SARS-CoV-2 nucleoprotein. However, sensitivity estimates can be strongly skewed by variations of the target concentrations within the clinical sample sets. Independent studies evaluating ATs from different manufacturers often display disparate target concentration distributions, making it difficult to compare sensitivity across products. We propose a new methodology to enhance the accuracy of sensitivity calculations, ensuring more reliable comparisons across ATs.
Methods
Sensitivity is estimated by modeling the probability of positive agreement (PPA) as a function of qRT-PCR cycle thresholds (Cts) via logistic regression of antigen test results. Raw sensitivity is calculated as the ratio of antigen test positives to total PCR positives. Adjusted sensitivity is derived by applying the PPA function to a reference concentration distribution, enabling uniform sensitivity comparisons across tests. This approach minimizes the impact of uneven sampling and external factors, as demonstrated using clinical data from a study in Chelsea, Massachusetts, USA.
Findings
Over two years, paired antigen and PCR positive tests from four AT suppliers were analyzed: A (211 tests), B (156), C (85), and D (43). The qRT-PCR Ct distributions varied, with suppliers A and D having more high viral load samples, while supplier C had more low viral load samples, causing significant discrepancies in raw sensitivity. Using the PPA function estimated from each supplier’s dataset, we calculated the corresponding adjusted sensitivities for common reference Ct distributions, highlighting how sample heterogeneity impacts raw sensitivity. Our approach successfully mitigates this variability, allowing for more accurate sensitivity comparisons.
Interpretation
This study demonstrates that sensitivity estimates from real-world data are susceptible to deviations caused by external factors, particularly the heterogeneity of qRT-PCR Ct distributions across studies. We present data supporting a novel methodology that adjusts for this variability by calculating the PPA function from raw data and determining the expected sensitivity based on a reference distribution of qRT-PCR Cts, allowing for more consistent and accurate sensitivity assessments.
Evidence before this study
Regulatory guidelines for antigen test (AT) performance generally require a balanced representation of low, mid, and high viral concentrations, though real-world sample distributions are highly variable. Previous studies often focus on sensitivity calculations, overlooking the impact of viral load distribution (Ct values) on results. Some studies use logistic regression to estimate the probability of positive agreement as a function of viral load, but no prior work has proposed adjusting sensitivity estimates based on a reference distribution of viral concentration.
Added value of the Study
This study presents a robust mathematical approach to adjust sensitivity estimates based on a standardized reference distribution of viral load, improving the precision of this key performance measure. By estimating the probability of positive agreement (PPA) as a function of viral load, we offer a more accurate assessment of AT product performance. We emphasize the importance of mitigating sample variability, showing how this method can enhance quality control and support regulatory oversight of antigen test performance.
Implications of all the available evidence
Our study underscores the limitations of calculating AT sensitivity directly from raw field data, which can lead to inaccurate evaluations. By applying our methodology, performance monitoring of ATs can be improved through standardized metrics, allowing for more reliable assessments. This approach helps both manufacturers and regulators establish clearer benchmarks for AT evaluation and comparison, addressing concerns about the sensitivity of antigen tests relative to gold-standard molecular methods. These improvements are critical for ensuring public confidence and regulatory accuracy in AT performance.
