<xhtml:span xmlns:xhtml="http://www.w3.org/1999/xhtml" xml:lang="en">Cut-off Cq value of GeneXpert in comparison to gold standard RT-PCR for diagnosis of SARS-CoV-2: A retrospective study </xhtml:span>
This article has been Reviewed by the following groups
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background The novel coronavirus disease (COVID-19) has highlighted vulnerabilities in healthcare systems and has single-handedly literally brought the world to a standstill. Diagnostic testing for COVID-19 is critical for understanding epidemiology, contact-tracing, case management, and to control the transmission of the SARS-CoV-2. RT-PCR test, the gold standard test, involves fairly complex steps and take nearly 24–48 h for generating the results. GeneXpert is a rapid nucleic-acid-detection-based test approved by ICMR (Indian Council of Medical Research), which can shorten the turnaround time significantly. Aim The aim of the study was to compare the performance of GeneXpert against the gold standard rRT-PCR for SARS-CoV-2. Materials and methods This retrospective study was conducted at a tertiary care centre from 25th March 2020 to 8th December 2020. The Nasopharyngeal/Oropharyngeal swabs that were sent to the VRDL laboratory for testing of SARS-CoV-2 by GeneXpert (CBNAAT) were included for the study. A total 220 samples positive and 50 samples negative for SARS-CoV-2 by GeneXpert were simultaneously tested for rRT-PCR. rRT-PCR was considered as gold standard test (reference) for calculating the sensitivity and specificity of GeneXpert (CBNAAT) test. Results Out of 220 samples positive for COVID-19 by GeneXpert, 118 (53.64%) were also positive by rRT-PCR while 102 (46.36%) showed negative results by rRT-PCR. However, 50 GeneXpert negative samples showed 100% agreement with rRT-PCR i.e. they were also negative by rRT-PCR. The GeneXpert sensitivity and specificity for COVID-19 was seen to be 100% (95% CI: 96.92 to 100%) and 32.89% (95% CI: 25.50 to 40.97%) respectively as compared to gold standard RT-PCR test. The positive predictive value (PPV) and negative predictive value (NPV) of GeneXpert for COVID-19 was found to be 53.64% and 100% respectively. Conclusion This study highlights that the GeneXpert test is highly sensitive and found to be helpful in emergency and challenging situations, where it can be useful in the early diagnosis and management of COVID-19 infection and thus, prevention of morbidity and mortality.
Article activity feed
-
Whilst the reviewer commends the authors for their efforts in improving this manuscript and has advised minor revisions, I have gone with a decision of major revisions as there several key issues remaining. Unless rectified this manuscript cannot be accepted for publication. Please work through the reviewer comments, paying particular attention to the way this manuscript is structured (centring the GeneXpert rather than the gold standard qPCR), removing the comparisons of Cq values which is scientifically unsound (this will include a title change to describe the findings rather a title which describes the work done), and working on the presentation of the data in Figure 2. The discussion needs to be improved to provide a more robust discussion of the data and the limitations of the GeneXpert which as the reviewer points out amounts to …
Whilst the reviewer commends the authors for their efforts in improving this manuscript and has advised minor revisions, I have gone with a decision of major revisions as there several key issues remaining. Unless rectified this manuscript cannot be accepted for publication. Please work through the reviewer comments, paying particular attention to the way this manuscript is structured (centring the GeneXpert rather than the gold standard qPCR), removing the comparisons of Cq values which is scientifically unsound (this will include a title change to describe the findings rather a title which describes the work done), and working on the presentation of the data in Figure 2. The discussion needs to be improved to provide a more robust discussion of the data and the limitations of the GeneXpert which as the reviewer points out amounts to a coin toss for its positive predictive value. Upon resubmission, please provide a tracked changes version of your revised manuscript so that the changes made are clear.
-
Comments to Author
The authors went to great efforts to improve the quality of the manuscript and followed the reviewer's advice. This is greatly appreciated. Minor Abstract * Remove word "literally" in the second line * Define VRDL * Total number of samples is 270 - 220 GeneXpert Positive and 50 GeneXpert Negative. This should be rectified all over, including on figure 1. Introduction * GeneXpert supplier can be added between parenthesis after the first time the test is mentioned. No need for a specific sentence for it. * Replace "till" by "until" in the third paragraph. Material and Methods * The sample size is 270 patients with suspected COVID-19 infection. The samples were then tested by two methods. * On the GeneXpert procedure section: rephrase the last sentence - the results are not processed by rRT-PCR, the …
Comments to Author
The authors went to great efforts to improve the quality of the manuscript and followed the reviewer's advice. This is greatly appreciated. Minor Abstract * Remove word "literally" in the second line * Define VRDL * Total number of samples is 270 - 220 GeneXpert Positive and 50 GeneXpert Negative. This should be rectified all over, including on figure 1. Introduction * GeneXpert supplier can be added between parenthesis after the first time the test is mentioned. No need for a specific sentence for it. * Replace "till" by "until" in the third paragraph. Material and Methods * The sample size is 270 patients with suspected COVID-19 infection. The samples were then tested by two methods. * On the GeneXpert procedure section: rephrase the last sentence - the results are not processed by rRT-PCR, the samples are. * On the rRT-PCR procedure section: Authors detail the target house keeping gene in the kit but not the two SARS-CoV-2 genes. I appreciate this information might not be included in the kit, but if it is would be helpful for the reader to have it easily available. * Last sentence of rRT-PCR procedure section the authors describe three categories for result interpretation: Positive, negative and inconclusive or invalid. What were the criteria used for a result to be included in each of the categories? These are also not mentioned further in the results or discussion so perhaps is irrelevant for the objective of the study. If that is the case, please remove the terminology or define which ones were included in the study. * Statistical analysis - since the population in this study was patients suspected of SARS-CoV-2 infection, it is valid to calculate disease prevalence with this data. Please remove every mention to that throughout. Major * Results are presented in a convoluted way. I would suggest following a simpler structure. Despite acknowledging the rRT-PCR is the golden standard and calculating the sensitivities and specificities considering so, the authors still structure their writing around the Gene-expert results, which made it incredibly difficult to follow. * The authors highlight the importance of using alternative rapid diagnostic methods in emergency settings, however that comes at a cost. They report a specificity of 32.89% and a positive predictive value of 53.64%. This means that only approximately half of the people with a positive test would have the disease. A positive GeneXpert result is essentially like tossing a coin according to your results. I think this needs further discussion in the appropriate section. * The results presented here suggest the use of GeneXpert is extremely helpful to exclude disease, as shown by the sensitivity and NPV of 100%. * I understand the authors are really keen in comparing the Cq values between the two methods. I need to highlight again the fact that these are two different qPCR assays. Even if similar chemistries are used, different assays with different targets, or same target but different primers and probes, will have different efficiencies. This will affect the raw Cq values making them not comparable. I can agree with the raw data being presented in the same graph if this is discussed and the authors draw caution to these conclusions. Figure 2: * Graph is very confusing and hard to follow. Axis titles need to be detailed in the graph and not just the legend. * Axis and legend font colour should not be grey. * In the method section it is mentioned that the cut-off value of 35 was used for rRT-PCR. Why do we have four data point above the cut-off? * Why is there so many red data points on the x axis, meaning Cq = 0? * I would suggest reformatting the graph and using a different presentation of data. The authors should consider presenting it as box and whisker plots, with each method in their own "box". The negative data points should be presented as the highest number of cycles used on that method (I presumed it would be 35 and 45 for rRT-PCR and GeneXpert respectively but unsure now, after seeing the red data points above 35). To distinguish from a positive result, the negatives can be represented with an open circle and that information should be given in the legend. The cut-offs for positivity could also be presented as a dashed horizontal line in the same colour used for the data points. Equally it's meaning should be detailed in the legend. Discussion * The discussion section would benefit from a better contextualisation of the results with the issue the authors are trying to address, at the start. * "Due to high N2 gene value with no E gene value, there are discordant results with positive Genexpert and negative rRT-PCR" this need to be clearer. I do nor understand what the authors mean here. The individual results for the two different targets was never mentioned before the discussion. If this is important enough to be discussed it needs to be presented. * The authors discus extensively the occurrence of false negatives, however they have not reported a single false negative. * They have, however, an issue with false positives: 46% of their GeneXpert results were false positives. It would be important to understand why and know the authors opinion: o Clinical information could help understand if these samples with rRT-PCR neg and GeneXpert + are in fact false positves or a methodology issue (eg. Lower cut-off, contamination of sample) o The differences in cut-offs used could also be affecting these results - the authors try to hint for this on table 2 when report that the PPV of GeneXpert would increase from 53.6% to 89.7% if Cq's
Please rate the manuscript for methodological rigour
Satisfactory
Please rate the quality of the presentation and structure of the manuscript
Poor
To what extent are the conclusions supported by the data?
Partially support
Do you have any concerns of possible image manipulation, plagiarism or any other unethical practices?
No
Is there a potential financial or other conflict of interest between yourself and the author(s)?
No
If this manuscript involves human and/or animal work, have the subjects been treated in an ethical manner and the authors complied with the appropriate guidelines?
Yes
-
-
Firstly, please accept my apologies for the severe delay in returning a decision on this manuscript. It has been extremely challenging to get two quality peer review reports on this important work. Both reviewers highlight significant concerns about the way this work has been carried out and presented that need to be addressed before publication can be approved. Please go through them carefully and address them point by point
-
Comments to Author
General comments: The authors raise a relevant question for the diagnosis of COVID-19, as it is extremely important to understand how different diagnostic tests compare to each other, especially when they are used interchangeable, as they were during the peak of the pandemic. The manuscript would benefit from more clarity in the language and rigour in the data presentation and interpretation. I welcome greatly the inclusion of raw qPCR data in appendix. I recommend the authors follow the MIQE guidelines (https://academic.oup.com/clinchem/article/55/4/611/5631762) throughout the manuscript. This will allow other to reproduce the findings described here. For example: * Use the standard nomenclature (Cq instead of Ct, RT-qPCR instead of RT-PCR or rRT-PCR) * Include Cq values for the negative …
Comments to Author
General comments: The authors raise a relevant question for the diagnosis of COVID-19, as it is extremely important to understand how different diagnostic tests compare to each other, especially when they are used interchangeable, as they were during the peak of the pandemic. The manuscript would benefit from more clarity in the language and rigour in the data presentation and interpretation. I welcome greatly the inclusion of raw qPCR data in appendix. I recommend the authors follow the MIQE guidelines (https://academic.oup.com/clinchem/article/55/4/611/5631762) throughout the manuscript. This will allow other to reproduce the findings described here. For example: * Use the standard nomenclature (Cq instead of Ct, RT-qPCR instead of RT-PCR or rRT-PCR) * Include Cq values for the negative controls and non-template controls (NTCs) - this is extremely important to understand the real effect of the different cut-offs (35 vs 45 Cq). * Include details of the extraction methods used. * Provide details of the assays: primer and probe sequences, fluorophore chosen, manufacturer of the oligos, concentrations of primers, probes, qPCR enzyme/mastermix used, other additives, thermocycling parameters, model and manufacturer of thermocycler used, amplicons length. If these details are not available - because these are commercial assays, and often the suppliers do not reveal this information, the authors must address this in the discussion. The authors correctly state that RT-qPCR is the golden standard method for COVID-19 diagnosis. However, throughout the manuscript, the authors seem to compare the qPCR results with the GeneXpert results, as if the latter was the reference method. For example, when they assume that 10% of the positivity was missed by the RT-qPCR. Instead, I would consider these false positives of the GeneXpert method. To fulfil the aim of the study I would suggest the authors to determine the sensitivity and specificity of the GeneExpert method, considering the positives and negatives obtained by RT-qPCR method true positives and true negatives. Discussion The authors raise the most import question here: how can we expect results from different methods to be comparable when using widely different cut-offs? They state it is "problematic" but I would go further and say it may not be scientifically correct. The authors compare Cq values obtained by two different methods. It is really tempting to do that, however this must not be done. A detected/ non-detected (or Pos/Neg) result can be compared, however raw Cq values cannot be compared between the two methods unless a standard curve of known quantities of RNA/cDNA is used in each run. There are a lot of factors that can lead to differences in the Cq's 1. different extraction methods 2. qPCR machine used. Was it the same for both methods? 2. more importantly, the assays are different (primers/ probe). 3. Samples were analysed at different times: storage can lead to degradation and yield different results even when the exact same assay and machine are used. Furthermore, even if all confounding factors mentioned above are removed allowing the data to be directly comparable, by looking at the raw data in appendix I am not fully convinced that the Cq are as similar as the authors state. Most ΔCq's (difference between the Cq's ) are greater than 1.5, for example, for sample 43 the ΔCq is 10. A 10-fold difference should yield a difference 3.3 Cq (assuming 100% efficiency). A ΔCq of 10 represents a 30-fold difference. I advise this sentence should be removed. The Geneexpert positives that are "missed" by qPCR need to be discussed further. This is either a result of the different cut-off Cq's applied or they are false positives or the sample was degraded during storage/transit. The storing conditions should also be mentioned in the method section. Inefficient extraction and presence of inhibitors would be similar across the two methods - assuming samples were extracted in parallel and then analysed using the two methods. Equipment malfunction would render the whole study void. Minor comments: Throughout: * Acronyms should be defined the first time they are being used. Abstract: * Line one: "Chinks in the armour" too colloquial. I suggest to rephrase, for example use "highlighted vulnerabilities in healthcare systems" * Line two: remove literally Introduction * Replace "around" by "approximately". * "Certain specific steps": can the author expand on which steps are slowing the process down? * Use containment level 2 (CL2) laboratory instead of BSL-2 and define the acronym the first time it is used. Results: * The tables are not easy to follow and it is not clear what they are trying to showcase exactly. What is the take home message? * Also, the pvalue presented is not clear what comparison was made. * The manuscript would benefit from a branched diagram explaining the sample populations and the positives/ negatives in each group.
Please rate the manuscript for methodological rigour
Poor
Please rate the quality of the presentation and structure of the manuscript
Satisfactory
To what extent are the conclusions supported by the data?
Partially support
Do you have any concerns of possible image manipulation, plagiarism or any other unethical practices?
No
Is there a potential financial or other conflict of interest between yourself and the author(s)?
No
If this manuscript involves human and/or animal work, have the subjects been treated in an ethical manner and the authors complied with the appropriate guidelines?
Yes
-
Comments to Author
William et al present a very short manuscript detailing the cut-off values of a new SARS-CoV-2 diagnostic technique (GeneXpert) compared to the global gold-standard of RT-PCR. This study was done in retrospect using a small cohort of 220 nasopharyngeal/oropharyngeal swabs taken from a defined period during the pandemic. Overall, the study has flaws that should be addressed if this were to be published. This information is useful however given that the likelihood of another pandemic is high and understanding different technology platforms will be useful information for future use. Below, the authors can find both major and minor comments I have regarding this paper. The major comments need to be addressed. Major comments: 1 - Why were only the positive GeneXpert samples tested by RT-PCR and not …
Comments to Author
William et al present a very short manuscript detailing the cut-off values of a new SARS-CoV-2 diagnostic technique (GeneXpert) compared to the global gold-standard of RT-PCR. This study was done in retrospect using a small cohort of 220 nasopharyngeal/oropharyngeal swabs taken from a defined period during the pandemic. Overall, the study has flaws that should be addressed if this were to be published. This information is useful however given that the likelihood of another pandemic is high and understanding different technology platforms will be useful information for future use. Below, the authors can find both major and minor comments I have regarding this paper. The major comments need to be addressed. Major comments: 1 - Why were only the positive GeneXpert samples tested by RT-PCR and not negative ones also? A true comparison to the gold standard would have tested all the samples on RT-PCR so as to ensure that you are not producing false-negatives using this new platform. Giving a patient a false-negative test has more significant consequences for infection control compared to a false-positive. The authors should consider testing these GeneXpert negative samples by RT-PCR to look at this. It would further strengthen the conclusions drawn from the data also and provide new data for analysis to improve the results section. 2 - Detection of high Ct value RT-PCR by GeneXpert does not necessarily mean that this virus is infectious. Work done by other reference laboratories showed that the likelihood of culturing infectious virus from a RT-PCR Ct value >35 was around 8 %. This is nicely shown in figure two of this link - https://doi.org/10.2807%2F1560-7917.ES.2020.25.32.2001483 Work to discriminate between cut-off Ct values and infectiousness should be considered for this new platform technology. This work naturally requires BSL-3 working conditions which is tricky and one appreciates this may not be possible. However, this correlation between positivity and infectiousness should be discussed. Moreover, high Ct values could be due to samples being taken at the very early stages of infection or poor swab technique by the patient. This can impact analytical sensitivity. This should imply that frequent testing of individuals is better than single point testing for the detection of positive individuals. This is also not discussed by the authors. 3 - These tables should be formatted better and the results could be expanded. The authors could consider plotting raw Ct values of the positives for GeneXpert Vs RT-PCR and looking at the relationships between the two in a more in-depth manner. 4 - The discussion is short and could be expanded significantly. how does this test compared to others on the market that have previously been evaluated. Minor comments: 1 - Abstract, background, line 3 - should be contact-tracing not contract. 2 - Consistency with nomenclature. Sometimes its SARS-CoV-2, others its SARS-COV-2 or SARS-COV2. This reviewer prefers the former, but consistency is key regardless of choice. 3 - RT-PCR procedure, paragraph 2, line 5. B-actin should use the Greek letter symbol. 4 - Consistency regarding rRT-PCR. there are numerous itinerartions of this also akin to point 2 above, please choose one and stick to it. 5 - detail how samples were stored following sample processing in the materials and methods
Please rate the manuscript for methodological rigour
Poor
Please rate the quality of the presentation and structure of the manuscript
Very poor
To what extent are the conclusions supported by the data?
Partially support
Do you have any concerns of possible image manipulation, plagiarism or any other unethical practices?
Yes: as described in the 'comments to the editor' there is a paragraph regarding statistical analysis that is a direct copy of another paper by one of the authors.
Is there a potential financial or other conflict of interest between yourself and the author(s)?
No
If this manuscript involves human and/or animal work, have the subjects been treated in an ethical manner and the authors complied with the appropriate guidelines?
Yes
-