Stringent thresholds in SARS-CoV-2 IgG assays lead to under-detection of mild infections
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Background
Thresholds for SARS-CoV-2 antibody assays have typically been determined using samples from symptomatic, often hospitalised, patients. In this setting the sensitivity and specificity of the best performing assays can both exceed 98%. However, antibody assay performance following mild infection is less clear.
Methods
We assessed quantitative IgG responses in a cohort of healthcare workers in Oxford, UK, with a high pre-test probability of Covid-19, in particular the 991/11,475(8.6%) who reported loss of smell/taste. We use anosmia/ageusia and other risk factors as probes for Covid-19 infection potentially undiagnosed by immunoassays by investigating their relationship with antibody readings either side of assay thresholds.
Results
The proportion of healthcare workers reporting anosmia/ageusia increased at antibody readings below diagnostic thresholds using an in-house ELISA ( n = 9324) and the Abbott Architect chemiluminescent microparticle immunoassay (CMIA; n = 11,324): 426/906 (47%) reported anosmia/ageusia with a positive ELISA, 59/449 (13.1%) with high-negative and 326/7969 (4.1%) with low-negative readings. Similarly, by CMIA, 518/1093 (47.4%) with a positive result reported anosmia/ageusia, 106/686 (15.5%) with a high-negative and 358/9563 (3.7%) with a low-negative result. Adjusting for the proportion of staff reporting anosmia/ageusia suggests the sensitivity of both assays in mild infection is lower than previously reported: Oxford ELISA 89.8% (95%CI 86.6–92.8%) and Abbott CMIA 79.3% (75.9–82.7%).
Conclusion
Following mild SARS-CoV-2 infection 10–30% of individuals may have negative immunoassay results. While lowered diagnostic thresholds may result in unacceptable specificity, our findings have implications for epidemiological analyses and result interpretation in individuals with a high pre-test probability. Samples from mild PCR-confirmed infections should be included in SARS-CoV-2 immunoassay evaluations.
Article activity feed
-
SciScore for 10.1101/2020.07.21.20159038: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement IRB: Deidentified data from staff testing were obtained from the Infections in Oxfordshire Research Database (IORD) which has generic Research Ethics Committee, Health Research Authority and Confidentiality Advisory Group approvals (19/SC/0403, 19/CAG/0144). Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources Serology for SARS-CoV-2 IgG to nucleocapsid protein was performed using the Abbott Architect i2000 chemiluminescent microparticle immunoassay (CMIA; Abbott, Maidenhead, UK). Abbott Architectsuggested: (Abbott ARCHITECT i1000sr System, …SciScore for 10.1101/2020.07.21.20159038: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement IRB: Deidentified data from staff testing were obtained from the Infections in Oxfordshire Research Database (IORD) which has generic Research Ethics Committee, Health Research Authority and Confidentiality Advisory Group approvals (19/SC/0403, 19/CAG/0144). Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources Serology for SARS-CoV-2 IgG to nucleocapsid protein was performed using the Abbott Architect i2000 chemiluminescent microparticle immunoassay (CMIA; Abbott, Maidenhead, UK). Abbott Architectsuggested: (Abbott ARCHITECT i1000sr System, RRID:SCR_019328)Abbottsuggested: (Abbott, RRID:SCR_010477)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:The lack of serially collected PCR data for all individuals in our study is another limitation. For example, we cannot say what proportion of those with equivocal antibody responses and anosmia/ageusia would have had a positive PCR test if tested shortly after infection. Similarly, serial antibody tests may have shown rising titres, although 99% of all those reporting anosmia/ageusia and providing a date of symptom onset were tested ≥14 days after symptom onset. Low initial antibody titres may also have waned by the point of testing too. A more general limitation is the absence of a gold standard for retrospective diagnosis of previous Covid-19. Follow up studies are needed to evaluate the extent to which individuals with symptoms or exposures strongly suggestive of Covid-19 such as anosmia or a PCR-confirmed household contact, but negative antibody results, have other evidence of infection, for example from T cell assays. Specific Covid-19 T cell responses have been reported in seronegative individuals who have been exposed to SARS-CoV-2.16 The sensitivity of SARS-CoV-2 serology in those with mild symptoms, i.e. the majority of the infected population, is likely to be lower than previously reported, i.e. 90% or less. However, even in the more extreme case that the sensitivity of serology for Covid-19 is as low as 60-70%, population estimates of previous infection in most settings would still be well below those needed for herd immunity, but still sufficiently different to cu...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
-
SciScore for 10.1101/2020.07.21.20159038: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement Deidentified data from staff testing were obtained from the Infections in Oxfordshire Research Database (IORD) which has generic Research Ethics Committee, Health Research Authority and Confidentiality Advisory Group approvals (19/SC/0403, 19/CAG/0144). Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable In a multivariable model, male gender was associated with fewer reports of loss of smell/taste (0.67 [0.50-0.89; p=0.007]) as was each 10-year increase in age (0.88 [0.79-0.99; p=0.03). Table 2: Resources
Antibodies Sentences Resources SARS-CoV-2 IgG antibody titres in convalescent symptomatic … SciScore for 10.1101/2020.07.21.20159038: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement Deidentified data from staff testing were obtained from the Infections in Oxfordshire Research Database (IORD) which has generic Research Ethics Committee, Health Research Authority and Confidentiality Advisory Group approvals (19/SC/0403, 19/CAG/0144). Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable In a multivariable model, male gender was associated with fewer reports of loss of smell/taste (0.67 [0.50-0.89; p=0.007]) as was each 10-year increase in age (0.88 [0.79-0.99; p=0.03). Table 2: Resources
Antibodies Sentences Resources SARS-CoV-2 IgG antibody titres in convalescent symptomatic healthcare workers SARS-CoV-2 IgGsuggested: NoneOn univariable analysis there was some evidence that junior doctors and laboratory staff with SARS-CoV-2 IgG antibodies were more likely to report loss of smell/taste (OR 1.99 [95%CI 1.06-3.75; p=0.03] and 3.11 [1.06-9.09; p=0.04] respectively). p=0.03suggested: NoneSimilar trends were seen for antibody titres and self-reported fever and myalgia (Supplementary Figures S1 and S2). S2suggested: NoneWedenote then umberofi ndivi dualswit hposi tiveant ibodyre sult sNpos and the numb erwit hhighneg ativ eresults Neqsuchtha t:!"#=∗$ %&($%&− '!& ()*+ ()/($% &($%&− '!&() *+ ()+(, ((,−'!&( )*+( ))Bo otstrapp ingwi th 1000iter ation swasusedt oe stimat etheuncert aintyina djust edsensit ivi tyr esults,acc ou ntingforva riationin pre viousl yre por tedsensit ivit ya swell asvaria tion aris ingfromt hepro por tionsf romthecur rentanalysis. − ' ! & ( )*+ ( ) + ( , ( ( , − ' ! & ( )*+ ( )suggested: NoneSoftware and Algorithms Sentences Resources Panel A shows the results using a trimeric spike ELISA and panel B the results from the Abbott CMIA targeting nucleocapsid protein, with blue showing results called negative and red showing those called as positive based on pre-defined assay thresholds. Abbottsuggested: (Abbott, SCR_010477)Serology for SARS-CoV-2 IgG to nucleocapsid protein was performed using the Abbott Architect i2000 chemiluminescent microparticle immunoassay (CMIA; Abbott, Maidenhead, UK). Abbott Architectsuggested: (Abbott ARCHITECT i1000sr System, SCR_018371)This work was supported by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Healthcare Associated Infections and Antimicrobial Resistance at Oxford University in partnership with Public Health England (PHE) [grant HPRU-201210041] and the NIHR Biomedical Research Centre, Oxford. NIHR Biomedical Research Centresuggested: NoneBDM is supported by the SGC, a registered charity (number 1097737) that receives funds from AbbVie, Bayer Pharma AG, Boehringer Ingelheim, Canada Foundation for Innovation, Eshelman Institute for Innovation, Genome Canada through Ontario Genomics Institute [OGI-055] AbbViesuggested: (AbbVie, SCR_010484)Data from additional tools added to each annotation on a weekly basis.
About SciScore
SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore is not a substitute for expert review. SciScore checks for the presence and correctness of RRIDs (research resource identifiers) in the manuscript, and detects sentences that appear to be missing RRIDs. SciScore also checks to make sure that rigor criteria are addressed by authors. It does this by detecting sentences that discuss criteria such as blinding or power analysis. SciScore does not guarantee that the rigor criteria that it detects are appropriate for the particular study. Instead it assists authors, editors, and reviewers by drawing attention to sections of the manuscript that contain or should contain various rigor criteria and key resources. For details on the results shown here, including references cited, please follow this link.
-
