Predicting infectivity: comparing four PCR‐based assays to detect culturable SARS‐CoV‐2 in clinical samples
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (Review Commons)
- Evaluated articles (ScreenIT)
Abstract
Article activity feed
-
-
Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.
Learn more at Review Commons
Reply to the reviewers
Reviewer #1 (Evidence, reproducibility and clarity (Required)):
**Summary:**
Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate).
Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 …
Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.
Learn more at Review Commons
Reply to the reviewers
Reviewer #1 (Evidence, reproducibility and clarity (Required)):
**Summary:**
Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate).
Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be more sensitive to infection than Vero E6, allowing a higher number of isolation from patient samples. They also showed their system to be more sensitive than a previously published sgE system as well as than a negative-strand RNA assay but less sensitive than the WHO/Charité primer/probe set. Anyway, all samples containing infectious particles (successful virus isolation on Vero E6-TMPRSS2) were detected with their primer/probe system contrary to the other tested sgE assay. They showed the negative strand assay to be unlikely to detect virus genetic material in samples which nevertheless contain infectious particles.
**Major comments:**
Are the key conclusions convincing?
I salute the intention of the authors to try to fix cut-off values for infectious patients but I would be more careful on the assertion of "using a total viral RNA Ct cut-off of >31 or specifically testing for sgRNA can serve as an effective rule-out test for viral infectivity". It is true that in this study, virus was not isolated from any of the samples below a Ct of 31 or negative in the developed sgE assay but all those assays are done on cell culture. We do not know how the transmission could occur for those samples from human to human. Being able to fix a cut-off in Ct value for a define PCR/RT-PCR system would be a great improvement for SARS-CoV-2 infected patient having to stay in quarantine. It is even more important for Ebola positive patients in Africa who has to stay in quarantine in precarious conditions under tents, warm temperatures and without privacy for long period because they still positive by RT-PCR. Unfortunately, fix those values would need a very high number of experiments, including animal experiment.
We appreciate the reviewer’s acknowledgment of the significance of this issue. We agree that in vivo animal experiments to more precisely determine the lowest infectious or transmissible dose would be valuable. But such experiments are outside the scope of the current study. To acknowledge the reviewer’s important point regarding the unavoidable limitations of cell culture systems, we have modified the abstract (line 51) to say “an effective rule out test for the presence of culturable virus,” a conclusion that is fully supported by our data.
Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether?
No
Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation.
No
Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments.
Yes.
Are the data and the methods presented in such a way that they can be reproduced?
Kinetic of SARS-CoV-2 (figure 2):
The method is not detailed in the Methods part and is not clear in the figure legend. When supernatant are collected, is it all the supernatant that is remove? An aliquot? If aliquot, do you replace with new medium?
We apologize for this omission and have included the requested details in the methods. We seed a separate well for each time point and collected the entire supernatant for a given time point, rather than replacing media. We added the following text to the methods section (lines 402-412): “Viral growth kinetics were measured in Vero E6 or Vero E6 TMPRSS2 cells at an MOI of 0.001. Separate wells were seeded for each time point, and growth curves were conducted in technical duplicates for each biological experiment. Supernatants and cell lysates were collected twice daily 1 & 2 dpi, and again on 3, 4, 7 and 8 dpi (Vero E6 TMPRSS2 cells were harvested for the final time at day 7 due to faster growth kinetics in this cell type). For each time point, the supernatant was removed and clarified to remove cellular debris, before being split into separate aliquots for RNA extraction (mixed 1:1 with AVE lysis buffer) and viral titration (by focus assay). Dead cells/debris that was pelleted after clarifying supernatants was combined with cells scraped from each well into PBS and spun again to obtain a pellet of all cell material from each timepoint. This pellet was then lysed in AVE viral lysis buffer for RNA extraction.”
Stability of infectious SARS-CoV-2:
I am very surprise by your results on stability of cultured virus, knowing we observed a decreased of SARS-CoV-2 titer in our lab after freezing/thawing steps. Do you freeze cell supernatant directly or do you prepare your samples another way? Please state it in the Methods part
We measured the stability after freeze/thaw for our normal high concentration viral stocks. Our viral stocks are grown in DMEM with 10% FBS, 1% HEPES, 1% pen/strep, and clarified before use. It is possible that lab-lab variation in the media components or HEPES concentration used to prepare viral stocks explains the differences seen in our work vs the reviewer’s lab. We have added the following additional detail to the methods section (lines 415-418) of the manuscript to clarify how these experiments were performed: “High concentration viral stocks (prepared as above in DMEM, 10% FBS, 1% HEPES, 1% pen/strep) were used to measure viral stability over time and after multiple freeze-thaw cycles. Stocks were stored at the indicated temperatures in the dark and aliquots were removed at the indicated days or after each freeze-thaw cycle for measuring infectious virus by focus assay.”
Are the experiments adequately replicated and statistical analysis adequate?
Yes
**Minor comments:**
Specific experimental issues that are easily addressable.
Figure 2C and D: Instead of Ct values in cells, it would be more relevant to normalize these results with an endogenous gene and present results as fold change to mock-infected cells. Because you affirm that the level of RNA decline than stay stable over the time but you also note there is CPE. If you have less cells but same level of viral RNA, it means you have an increase in the RNA level in alive cells.
We have measured the GAPDH level in these cells over time, and that data is included as gray lines in Fig 2 C&D (see new figure 2). As we are combining the cell pellet from clarified supernatants with the cells that remain adherent to the dish for each harvested timepoint we expect to be harvesting the majority of cells/cell debris for each time point. The levels of GAPDH remain broadly similar over the viral growth curve, with no drop in RNA levels.
It would have been interesting to have the results of isolation at different time-point of treatment for patient samples (figure 3A and B) to see if the virus is stable in samples
We have access to only limited volume (several hundred µl) of residual patient sample which would make it technically challenging to compare multiple days of storage conditions/ temperatures. Unfortunately, we do not have any remaining sample volume for the specimens used in this study, and so we are unable to perform additional isolations at other times/temperatures. While we agree this would be an interesting line of future inquiry, we feel it is outside the scope of the current study.
Are prior studies referenced appropriately? Yes
Are the text and figures clear and accurate?
Yes.
Line 140: "this delay in virus and RNA production". You do not talk about RNA yet...
We have removed “and RNA” from this sentence and replaced with “infectious virus production”.
Line 156 to 163: sgE RNA detected in cell free supernatant. Can't it come from lysed cells?
We have replaced “cell-free” with “clarified”.
Line 167: "...virus in cell culture time course experiment in TMPRRS2 expressing cells (fig.2)"
We have modified this text to read according to the Reviewer’s suggestion.
Ligne 258: Fig 6A and B
We have added the missing reference to Fig 6B as requested.
-Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No
Reviewer #1 (Significance (Required)):
Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.
This new primer/probe system will participate to the accurate diagnostic of SARS-CoV-2. The comparison with the existing methods is relevant to highlight the strengths and weaknesses of each system. Comparison of isolation of SARS-CoV-2 on commonly used Vero E6 with Vero E6-TMPRSS2 will lead to a great improvement of the isolation method for SARS-CoV-2.
We appreciate the Reviewer’s assessment of the significance of our study and the improvement in our isolation method compared to the existing standard of using Vero E6 cells.
Place the work in the context of the existing literature (provide references, where appropriate).
Properly done in the introduction of the paper.
State what audience might be interested in and influenced by the reported findings.
Diagnostic laboratories
Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.
Virology, Molecular Biology, cell biology
Not enough expertise to evaluate ROC data/analysis
Reviewer #2 (Evidence, reproducibility and clarity (Required)):
**Summary:**
Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay and conducted comparisons with other assays on the same clinical samples. There are some areas that the authors should address prior to publication.
**Major comments:**
The authors repeatedly tout VeroE6 TMRSS2 cells as supporting higher viral infection. Therefore, the authors should address why one clinical isolate (E16) was culturable in VeroE6 but not VeroE6 TMRSS2. Was this experiment repeated multiple times? What are the reasons for this discrepancy?
We did not have sufficient residual sample volume to repeat isolation attempts of any clinical specimen, so we are limited to a single data point for each cell line. It is possible that this sample had levels of infectious virus at the limit of detection, and stochastic probability meant infectious virus was only present in the aliquot used to infect the Vero E6 (rather than Vero E6-TMPRSS2) cells. It is also possible that viral adaptation/evolution occurred in the VeroE6 well that allowed this virus to successfully grow, but we do not have sequencing data or remaining nucleic acids to test this theory.
The authors' argument at lines 166-169 is not supported by the data in Fig. 2. The levels of viral RNA between VeroE6 and VeroE6 TMRSS2 appear to show similar trends in the supernatant across the time course but the infectious viral levels are dramatically different. This discordance between FFU levels and RNA levels cannot be explained by instability of viral particles alone. Have the authors looked into differences in viral particles produced from these two cell lines? The authors should collect virus particles from these two cell lines and conduct the stability experiment in Fig 2D to directly test the hypothesis that indeed the drop seen in FFU in VeroE6 TMRSS2 is due to instability.
We apologize for the confusion. We did not intend to make claims about differences in particle stability as a result of the cell line used for viral production, but rather to highlight a general observation that RNA was more stable than infectious virus. This is more obvious in the TMRPSS2 cell line, as replication is faster and more synchronized than in Vero E6 cells (the TMRPSS2 cells are largely dead by day 4, whereas infection progresses more slowly in Vero E6 cells so that new virions continue to be produced during the measured time period). We have added clarifying text at line 167-169, *“We observed that SARS-CoV-2 RNA species persist for much longer than infectious virus in cell culture time course experiments, a feature that was most obvious in Vero E6 TMRPSS-2 cells due to their viral kinetics but is likely not cell specific (Fig 2).” *
The evidence for the packaging of sgE RNA into virions is weak. GAPDH detection by PCR is not a proof that the concentration process did not pellet RNA nonspecifically. First, the authors should provide ample information about viral isolation process at line 379 including rotor, centrifuge and speed utilized. In addition, ribosomes typically stay intact following viral lysis (and can be found in supernatant after release from dead cells). Actively translating ribosomes can contain sgE RNA as well. The authors should consider detecting ribosomal RNAs in their samples to rule out the possibility of contaminating ribosomes. In addition, the authors should strongly consider repeating the experiment with high EDTA concentration to break up ribosomes and only pellet virions.
We have added additional experimental details (rotor, centrifuge and speed) describing how the viral concentration step was performed (line 389-394), “Viral RNA (courtesy of David Bauer, The Francis Crick Institute, UK) from concentrated SARS-CoV-2 (England02 strain, B lineage ‘Wuhan-like’) was obtained by clarifying viral supernatants (2 x 4000 rpm for 30 mins at 4°C in a Beckman Allegra X-30R centrifuge with a SX4400 rotor), overlaying clarified media onto a 30% sucrose/PBS cushion (1/4th tube volume) and concentrating by ultracentrifugation in a Beckman ultra XPN-90 centrifuge with SW32TI rotor for 90 min at 25,500 rpm at 4°C. Pellets were then resuspended in buffer and extracted with TRIzol LS.” We thank the reviewer for their suggestion of including an additional control, and we have added an 18S primer-probe set (see new Figure 8). This data, while not as pronounced as the GAPDH control, suggests that the ultracentrifugation step has removed significant amounts of 18S RNA (though the clarified supernatants retain similar amounts of 18S RNA as the cells, suggesting that clarification alone is not sufficient to remove contaminating ribosomes). While we agree that repeating the ultracentrifuge concentration with high concentrations of EDTA is an interesting line of inquiry we feel it is outside the scope of this manuscript (and we face additional technical restrictions to pursue this as we currently lack access to an ultracentrifuge at BSL-3). We have updated the discussion to include the possibility of residual ribosome-protected fragments of sgE as a potential alternative interpretation (line 350-352).
**Minor comments:**
At line 197, the authors refer to "viruses" with lower levels of SARS-CoV2 RNA. This is incorrect and should be changed to "isolates" as the SARS-CoV2 virus particle does not package variable amount of genomic RNA.
We have changed this to “clinical specimens” for clarity.
The authors statement on lines 210-212 does not seem to be supported clearly by Fig. 5. The authors should consider including trendlines as well as other analyses that help show the correlation between viral RNA vs FFU. In addition, the authors should label the Y-axis clearly for Fig. 5.
We have added clarifying labels to both the X and Y axes. Due to the limited sample volume we were unable to directly measure the infectious titers from the clinical samples used in this study, and thus the FFU/mL represents the titer post-isolation while the CT represents the amount of RNA pre-isolation. Nonetheless, we do see broad trends (ie, the colored dots are generally arranged in rainbow order from left to right, though we agree there is variation within this trend). We have also modified the text at lines 212-217 to reflect the reviewer’s concern- “Greater initial viral RNA levels was broadly associated with faster viral growth in both cell lines (seen in the progression of colors from left to right), however we saw significant variation within these trends. Our data suggests that when standard SARS-CoV-2 RNA RT-PCR values are the only available data for patient or population-level viral loads, they are useful in gauging the presence of infectious virus in patient NP samples (Fig 5).”
The authors should expand on the methodology for creating ROC curves at line 467.
We have included the following text in the methods section for ROC curve analysis:
“ROC curves were generated using R and plotted with the ggplot2 package
__ [43]. For each potential scoring marker (CT_e, CT_sge1, CT_sge2, neg_e,) samples were ordered by that marker, followed by culturable status. The false-positive rate was calculated as the cumulative count of culturable samples (after ordering by marker intensity) divided by the total count of culturable samples; the true positive rate was calculated as the cumulative count of non-culturable samples (after ordering) divided by the total count of non-culturable samples. The false positive rate was plotted on the X axis of the ROC curves and the true positive rate on the Y axis.”__
Reviewer #2 (Significance (Required)):
This study is significant because it assesses the utility of several clinical assays for the measurement of viral RNA and correlating it with culturable virus. This is important in the field because it helps to identify methods whereby infectivity can be predicted from a simple diagnostic test. This is important to know as a virologist working in the SARS-CoV2 field. It is also important from a public health perspective to better define quarantine requirements for persons testing positive. While the study provided a new primer probe set, it appears that the already available WHO total E RNA assay is superior in predicting infectivity and this study provides further evidence to support this notion.
We appreciate the Reviewer’s assessment that this study is significant and provides information of high interest to SARS-CoV-2 virologists that also has important public health implications.
Reviewer #1 (Evidence, reproducibility and clarity (Required)):
**Summary:**
Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be more sensitive to infection than Vero E6, allowing a higher number of isolation from patient samples. They also showed their system to be more sensitive than a previously published sgE system as well as than a negative-strand RNA assay but less sensitive than the WHO/Charité primer/probe set. Anyway, all samples containing infectious particles (successful virus isolation on Vero E6-TMPRSS2) were detected with their primer/probe system contrary to the other tested sgE assay. They showed the negative strand assay to be unlikely to detect virus genetic material in samples which nevertheless contain infectious particles.
**Major comments:**
Are the key conclusions convincing?
I salute the intention of the authors to try to fix cut-off values for infectious patients but I would be more careful on the assertion of "using a total viral RNA Ct cut-off of >31 or specifically testing for sgRNA can serve as an effective rule-out test for viral infectivity". It is true that in this study, virus was not isolated from any of the samples below a Ct of 31 or negative in the developed sgE assay but all those assays are done on cell culture. We do not know how the transmission could occur for those samples from human to human. Being able to fix a cut-off in Ct value for a define PCR/RT-PCR system would be a great improvement for SARS-CoV-2 infected patient having to stay in quarantine. It is even more important for Ebola positive patients in Africa who has to stay in quarantine in precarious conditions under tents, warm temperatures and without privacy for long period because they still positive by RT-PCR. Unfortunately, fix those values would need a very high number of experiments, including animal experiment.
We appreciate the reviewer’s acknowledgment of the significance of this issue. We agree that *in vivo *animal experiments to more precisely determine the lowest infectious or transmissible dose would be valuable. But such experiments are outside the scope of the current study. To acknowledge the reviewer’s important point regarding the unavoidable limitations of cell culture systems, we have modified the abstract (line 51) to say “an effective rule out test for the presence of culturable virus,” a conclusion that is fully supported by our data.
Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? No
Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. No
Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. Yes.
Are the data and the methods presented in such a way that they can be reproduced?
Kinetic of SARS-CoV-2 (figure 2): The method is not detailed in the Methods part and is not clear in the figure legend. When supernatant are collected, is it all the supernatant that is remove? An aliquot? If aliquot, do you replace with new medium?
We apologize for this omission and have included the requested details in the methods. We seed a separate well for each time point and collected the entire supernatant for a given time point, rather than replacing media. We added the following text to the methods section (lines 402-412): “Viral growth kinetics were measured in Vero E6 or Vero E6 TMPRSS2 cells at an MOI of 0.001. Separate wells were seeded for each time point, and growth curves were conducted in technical duplicates for each biological experiment. Supernatants and cell lysates were collected twice daily 1 & 2 dpi, and again on 3, 4, 7 and 8 dpi (Vero E6 TMPRSS2 cells were harvested for the final time at day 7 due to faster growth kinetics in this cell type). For each time point, the supernatant was removed and clarified to remove cellular debris, before being split into separate aliquots for RNA extraction (mixed 1:1 with AVE lysis buffer) and viral titration (by focus assay). Dead cells/debris that was pelleted after clarifying supernatants was combined with cells scraped from each well into PBS and spun again to obtain a pellet of all cell material from each timepoint. This pellet was then lysed in AVE viral lysis buffer for RNA extraction.”
Stability of infectious SARS-CoV-2: I am very surprise by your results on stability of cultured virus, knowing we observed a decreased of SARS-CoV-2 titer in our lab after freezing/thawing steps. Do you freeze cell supernatant directly or do you prepare your samples another way? Please state it in the Methods part
We measured the stability after freeze/thaw for our normal high concentration viral stocks. Our viral stocks are grown in DMEM with 10% FBS, 1% HEPES, 1% pen/strep, and clarified before use. It is possible that lab-lab variation in the media components or HEPES concentration used to prepare viral stocks explains the differences seen in our work vs the reviewer’s lab. We have added the following additional detail to the methods section (lines 415-418) of the manuscript to clarify how these experiments were performed: “High concentration viral stocks (prepared as above in DMEM, 10% FBS, 1% HEPES, 1% pen/strep) were used to measure viral stability over time and after multiple freeze-thaw cycles. Stocks were stored at the indicated temperatures in the dark and aliquots were removed at the indicated days or after each freeze-thaw cycle for measuring infectious virus by focus assay.”
Are the experiments adequately replicated and statistical analysis adequate? Yes
**Minor comments:**
Specific experimental issues that are easily addressable.
Figure 2C and D: Instead of Ct values in cells, it would be more relevant to normalize these results with an endogenous gene and present results as fold change to mock-infected cells. Because you affirm that the level of RNA decline than stay stable over the time but you also note there is CPE. If you have less cells but same level of viral RNA, it means you have an increase in the RNA level in alive cells.
We have measured the GAPDH level in these cells over time, and that data is included as gray lines in Fig 2 C&D (see updated figure). As we are combining the cell pellet from clarified supernatants with the cells that remain adherent to the dish for each harvested timepoint we expect to be harvesting the majority of cells/cell debris for each time point. The levels of GAPDH remain broadly similar over the viral growth curve, with no drop in RNA levels.
It would have been interesting to have the results of isolation at different time-point of treatment for patient samples (figure 3A and B) to see if the virus is stable in samples
We have access to only limited volume (several hundred µl) of residual patient sample which would make it technically challenging to compare multiple days of storage conditions/ temperatures. Unfortunately, we do not have any remaining sample volume for the specimens used in this study, and so we are unable to perform additional isolations at other times/temperatures. While we agree this would be an interesting line of future inquiry, we feel it is outside the scope of the current study.
Are prior studies referenced appropriately? Yes
Are the text and figures clear and accurate? Yes.
Line 140: "this delay in virus and RNA production". You do not talk about RNA yet...
We have removed “and RNA” from this sentence and replaced with “infectious virus production”.
Line 156 to 163: sgE RNA detected in cell free supernatant. Can't it come from lysed cells?
We have replaced “cell-free” with “clarified”.
Line 167: "...virus in cell culture time course experiment in TMPRRS2 expressing cells (fig.2)"
We have modified this text to read according to the Reviewer’s suggestion.
Ligne 258: Fig 6A and B
We have added the missing reference to Fig 6B as requested.
-Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No
Reviewer #1 (Significance (Required)):
Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.
This new primer/probe system will participate to the accurate diagnostic of SARS-CoV-2. The comparison with the existing methods is relevant to highlight the strengths and weaknesses of each system. Comparison of isolation of SARS-CoV-2 on commonly used Vero E6 with Vero E6-TMPRSS2 will lead to a great improvement of the isolation method for SARS-CoV-2.
We appreciate the Reviewer’s assessment of the significance of our study and the improvement in our isolation method compared to the existing standard of using Vero E6 cells.
Place the work in the context of the existing literature (provide references, where appropriate). Properly done in the introduction of the paper.
State what audience might be interested in and influenced by the reported findings. Diagnostic laboratories
Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate. Virology, Molecular Biology, cell biology Not enough expertise to evaluate ROC data/analysis
__Reviewer #2 __(Evidence, reproducibility and clarity (Required)):
**Summary:**
Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay and conducted comparisons with other assays on the same clinical samples. There are some areas that the authors should address prior to publication.
**Major comments:**
The authors repeatedly tout VeroE6 TMRSS2 cells as supporting higher viral infection. Therefore, the authors should address why one clinical isolate (E16) was culturable in VeroE6 but not VeroE6 TMRSS2. Was this experiment repeated multiple times? What are the reasons for this discrepancy?
We did not have sufficient residual sample volume to repeat isolation attempts of any clinical specimen, so we are limited to a single data point for each cell line. It is possible that this sample had levels of infectious virus at the limit of detection, and stochastic probability meant infectious virus was only present in the aliquot used to infect the Vero E6 (rather than Vero E6-TMPRSS2) cells. It is also possible that viral adaptation/evolution occurred in the VeroE6 well that allowed this virus to successfully grow, but we do not have sequencing data or remaining nucleic acids to test this theory.
The authors' argument at lines 166-169 is not supported by the data in Fig. 2. The levels of viral RNA between VeroE6 and VeroE6 TMRSS2 appear to show similar trends in the supernatant across the time course but the infectious viral levels are dramatically different. This discordance between FFU levels and RNA levels cannot be explained by instability of viral particles alone. Have the authors looked into differences in viral particles produced from these two cell lines? The authors should collect virus particles from these two cell lines and conduct the stability experiment in Fig 2D to directly test the hypothesis that indeed the drop seen in FFU in VeroE6 TMRSS2 is due to instability.
We apologize for the confusion. We did not intend to make claims about differences in particle stability as a result of the cell line used for viral production, but rather to highlight a general observation that RNA was more stable than infectious virus. This is more obvious in the TMRPSS2 cell line, as replication is faster and more synchronized than in Vero E6 cells (the TMRPSS2 cells are largely dead by day 4, whereas infection progresses more slowly in Vero E6 cells so that new virions continue to be produced during the measured time period). We have added clarifying text at line 167-169, “*We observed that SARS-CoV-2 RNA species persist for much longer than infectious virus in cell culture time course experiments, a feature that was most obvious in Vero E6 TMRPSS-2 cells due to their viral kinetics but is likely not cell specific (Fig 2).” *
The evidence for the packaging of sgE RNA into virions is weak. GAPDH detection by PCR is not a proof that the concentration process did not pellet RNA nonspecifically. First, the authors should provide ample information about viral isolation process at line 379 including rotor, centrifuge and speed utilized. In addition, ribosomes typically stay intact following viral lysis (and can be found in supernatant after release from dead cells). Actively translating ribosomes can contain sgE RNA as well. The authors should consider detecting ribosomal RNAs in their samples to rule out the possibility of contaminating ribosomes. In addition, the authors should strongly consider repeating the experiment with high EDTA concentration to break up ribosomes and only pellet virions.
We have added additional experimental details (rotor, centrifuge and speed) describing how the viral concentration step was performed (line 389-394), “Viral RNA (courtesy of David Bauer, The Francis Crick Institute, UK) from concentrated SARS-CoV-2 (England02 strain, B lineage ‘Wuhan-like’) was obtained by clarifying viral supernatants (2 x 4000 rpm for 30 mins at 4°C in a Beckman Allegra X-30R centrifuge with a SX4400 rotor), overlaying clarified media onto a 30% sucrose/PBS cushion (1/4th tube volume) and concentrating by ultracentrifugation in a Beckman ultra XPN-90 centrifuge with SW32TI rotor for 90 min at 25,500 rpm at 4°C. Pellets were then resuspended in buffer and extracted with TRIzol LS.” We thank the reviewer for their suggestion of including an additional control, and we have added an 18S primer-probe set (see new Figure 8). This data, while not as pronounced as the GAPDH control, suggests that the ultracentrifugation step has removed significant amounts of 18S RNA (though the clarified supernatants retain similar amounts of 18S RNA as the cells, suggesting that clarification alone is not sufficient to remove contaminating ribosomes). While we agree that repeating the ultracentrifuge concentration with high concentrations of EDTA is an interesting line of inquiry we feel it is outside the scope of this manuscript (and we face additional technical restrictions to pursue this as we currently lack access to an ultracentrifuge at BSL-3). We have updated the discussion to include the possibility of residual ribosome-protected fragments of sgE as a potential alternative interpretation (line 350-352).
**Minor comments:**
At line 197, the authors refer to "viruses" with lower levels of SARS-CoV2 RNA. This is incorrect and should be changed to "isolates" as the SARS-CoV2 virus particle does not package variable amount of genomic RNA.
We have changed this to “clinical specimens” for clarity.
The authors statement on lines 210-212 does not seem to be supported clearly by Fig. 5. The authors should consider including trendlines as well as other analyses that help show the correlation between viral RNA vs FFU. In addition, the authors should label the Y-axis clearly for Fig. 5.
We have added clarifying labels to both the X and Y axes. Due to the limited sample volume we were unable to directly measure the infectious titers from the clinical samples used in this study, and thus the FFU/mL represents the titer post-isolation while the CT represents the amount of RNA pre-isolation. Nonetheless, we do see broad trends (ie, the colored dots are generally arranged in rainbow order from left to right, though we agree there is variation within this trend). We have also modified the text at lines 212-217 to reflect the reviewer’s concern- “Greater initial viral RNA levels was broadly associated with faster viral growth in both cell lines (seen in the progression of colors from left to right), however we saw significant variation within these trends. Our data suggests that when standard SARS-CoV-2 RNA RT-PCR values are the only available data for patient or population-level viral loads, they are useful in gauging the presence of infectious virus in patient NP samples (Fig 5).”
The authors should expand on the methodology for creating ROC curves at line 467.
We have included the following text in the methods section for ROC curve analysis:
“ROC curves were generated using R [43]. For each potential scoring marker (CT_e, CT_sge1, CT_sge2, neg_e,) samples were ordered by that marker, followed by culturable status. The false-positive rate was calculated as the cumulative count of culturable samples (after ordering by marker intensity) divided by the total count of culturable samples; the true positive rate was calculated as the cumulative count of non-culturable samples (after ordering) divided by the total count of non-culturable samples. The false positive rate was plotted on the X axis of the ROC curves and the true positive rate on the Y axis.”
Reviewer #2 (Significance (Required)):
This study is significant because it assesses the utility of several clinical assays for the measurement of viral RNA and correlating it with culturable virus. This is important in the field because it helps to identify methods whereby infectivity can be predicted from a simple diagnostic test. This is important to know as a virologist working in the SARS-CoV2 field. It is also important from a public health perspective to better define quarantine requirements for persons testing positive. While the study provided a new primer probe set, it appears that the already available WHO total E RNA assay is superior in predicting infectivity and this study provides further evidence to support this notion.
We appreciate the Reviewer’s assessment that this study is significant and provides information of high interest to SARS-CoV-2 virologists that also has important public health implications.
-
Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.
Learn more at Review Commons
Referee #2
Evidence, reproducibility and clarity
Summary:
Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay …
Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.
Learn more at Review Commons
Referee #2
Evidence, reproducibility and clarity
Summary:
Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay and conducted comparisons with other assays on the same clinical samples. There are some areas that the authors should address prior to publication.
Major comments:
-The authors repeatedly tout VeroE6 TMRSS2 cells as supporting higher viral infection. Therefore, the authors should address why one clinical isolate (E16) was culturable in VeroE6 but not VeroE6 TMRSS2. Was this experiment repeated multiple times? What are the reasons for this discrepancy?
-The authors' argument at lines 166-169 is not supported by the data in Fig. 2. The levels of viral RNA between VeroE6 and VeroE6 TMRSS2 appear to show similar trends in the supernatant across the time course but the infectious viral levels are dramatically different. This discordance between FFU levels and RNA levels cannot be explained by instability of viral particles alone. Have the authors looked into differences in viral particles produced from these two cell lines? The authors should collect virus particles from these two cell lines and conduct the stability experiment in Fig 2D to directly test the hypothesis that indeed the drop seen in FFU in VeroE6 TMRSS2 is due to instability.
-The evidence for the packaging of sgE RNA into virions is weak. GAPDH detection by PCR is not a proof that the concentration process did not pellet RNA nonspecifically. First, the authors should provide ample information about viral isolation process at line 379 including rotor, centrifuge and speed utilized. In addition, ribosomes typically stay intact following viral lysis (and can be found in supernatant after release from dead cells). Actively translating ribosomes can contain sgE RNA as well. The authors should consider detecting ribosomal RNAs in their samples to rule out the possibility of contaminating ribosomes. In addition, the authors should strongly consider repeating the experiment with high EDTA concentration to break up ribosomes and only pellet virions.
Minor comments:
-At line 197, the authors refer to "viruses" with lower levels of SARS-CoV2 RNA. This is incorrect and should be changed to "isolates" as the SARS-CoV2 virus particle does not package variable amount of genomic RNA.
-The authors statement on lines 210-212 does not seem to be supported clearly by Fig. 5. The authors should consider including trendlines as well as other analyses that help show the correlation between viral RNA vs FFU. In addition, the authors should label the Y-axis clearly for Fig. 5.
-The authors should expand on the methodology for creating ROC curves at line 467.
Significance
This study is significant because it assesses the utility of several clinical assays for the measurement of viral RNA and correlating it with culturable virus. This is important in the field because it helps to identify methods whereby infectivity can be predicted from a simple diagnostic test. This is important to know as a virologist working in the SARS-CoV2 field. It is also important from a public health perspective to better define quarantine requirements for persons testing positive. While the study provided a new primer probe set, it appears that the already available WHO total E RNA assay is superior in predicting infectivity and this study provides further evidence to support this notion.
-
Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.
Learn more at Review Commons
Referee #1
Evidence, reproducibility and clarity
Summary:
Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be …
Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.
Learn more at Review Commons
Referee #1
Evidence, reproducibility and clarity
Summary:
Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be more sensitive to infection than Vero E6, allowing a higher number of isolation from patient samples. They also showed their system to be more sensitive than a previously published sgE system as well as than a negative-strand RNA assay but less sensitive than the WHO/Charité primer/probe set. Anyway, all samples containing infectious particles (successful virus isolation on Vero E6-TMPRSS2) were detected with their primer/probe system contrary to the other tested sgE assay. They showed the negative strand assay to be unlikely to detect virus genetic material in samples which nevertheless contain infectious particles.
Major comments:
-Are the key conclusions convincing?
I salute the intention of the authors to try to fix cut-off values for infectious patients but I would be more careful on the assertion of "using a total viral RNA Ct cut-off of >31 or specifically testing for sgRNA can serve as an effective rule-out test for viral infectivity". It is true that in this study, virus was not isolated from any of the samples below a Ct of 31 or negative in the developed sgE assay but all those assays are done on cell culture. We do not know how the transmission could occur for those samples from human to human. Being able to fix a cut-off in Ct value for a define PCR/RT-PCR system would be a great improvement for SARS-CoV-2 infected patient having to stay in quarantine. It is even more important for Ebola positive patients in Africa who has to stay in quarantine in precarious conditions under tents, warm temperatures and without privacy for long period because they still positive by RT-PCR. Unfortunately, fix those values would need a very high number of experiments, including animal experiment.
-Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? No
-Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. No
-Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. Yes.
-Are the data and the methods presented in such a way that they can be reproduced?
-Kinetic of SARS-CoV-2 (figure 2): The method is not detailed in the Methods part and is not clear in the figure legend. When supernatant are collected, is it all the supernatant that is remove? An aliquot? If aliquot, do you replace with new medium? -Stability of infectious SARS-CoV-2: I am very surprise by your results on stability of cultured virus, knowing we observed a decreased of SARS-CoV-2 titer in our lab after freezing/thawing steps. Do you freeze cell supernatant directly or do you prepare your samples another way? Please state it in the Methods part
-Are the experiments adequately replicated and statistical analysis adequate? Yes
Minor comments:
- Specific experimental issues that are easily addressable.
Figure 2C and D: Instead of Ct values in cells, it would be more relevant to normalize these results with an endogenous gene and present results as fold change to mock-infected cells. Because you affirm that the level of RNA decline than stay stable over the time but you also note there is CPE. If you have less cells but same level of viral RNA, it means you have an increase in the RNA level in alive cells. It would have been interesting to have the results of isolation at different time-point of treatment for patient samples (figure 3A and B) to see if the virus is stable in samples
-Are prior studies referenced appropriately? Yes
-Are the text and figures clear and accurate? Yes.
Line 140: "this delay in virus and RNA production". You do not talk about RNA yet...
Line 156 to 163: sgE RNA detected in cell free supernatant. Can't it come from lysed cells?
Line 167: "...virus in cell culture time course experiment in TMPRRS2 expressing cells (fig.2)"
Ligne 258: Fig 6A and B
-Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No
Significance
-Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.
This new primer/probe system will participate to the accurate diagnostic of SARS-CoV-2. The comparison with the existing methods is relevant to highlight the strengths and weaknesses of each system. Comparison of isolation of SARS-CoV-2 on commonly used Vero E6 with Vero E6-TMPRSS2 will lead to a great improvement of the isolation method for SARS-CoV-2.
-Place the work in the context of the existing literature (provide references, where appropriate). Properly done in the introduction of the paper.
-State what audience might be interested in and influenced by the reported findings. Diagnostic laboratories
-Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate. Virology, Molecular Biology, cell biology Not enough expertise to evaluate ROC data/analysis
-
SciScore for 10.1101/2021.07.14.21260544: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: Viruses and cells: The use of deidentified positive specimens for the above studies was approved by the University of Washington Institutional Review Board (STUDY00010205). Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Cell Line Authentication not detected. Table 2: Resources
Antibodies Sentences Resources Cells were permeabilized with 0.1% 100X Triton in 1× PBS for 15 min and then incubated with a primary, cross-reactive rabbit anti-SARS-CoV N monoclonal antibody (Sinobiological, distributed by Thermo Fisher, Cat. #40143-R001 at a dilution of 1:20,000) followed by a peroxidase-labelled goat anti-rabbit antibody (SeraCare, … SciScore for 10.1101/2021.07.14.21260544: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics IRB: Viruses and cells: The use of deidentified positive specimens for the above studies was approved by the University of Washington Institutional Review Board (STUDY00010205). Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Cell Line Authentication not detected. Table 2: Resources
Antibodies Sentences Resources Cells were permeabilized with 0.1% 100X Triton in 1× PBS for 15 min and then incubated with a primary, cross-reactive rabbit anti-SARS-CoV N monoclonal antibody (Sinobiological, distributed by Thermo Fisher, Cat. #40143-R001 at a dilution of 1:20,000) followed by a peroxidase-labelled goat anti-rabbit antibody (SeraCare, Milford, MA, USA, Cat. #5220-0336 diluted to 1:2,000) and then the peroxidase substrate (SeraCare, Cat. #5510-0030). anti-SARS-CoV Nsuggested: Noneanti-rabbitsuggested: (SeraCare KPL Cat# 5220-0336, RRID:AB_2857917)Experimental Models: Cell Lines Sentences Resources Vero E6 or Vero E6-TMPRSS2 cells were seeded in diagonally adjacent wells of 24 well plates (12 wells seeded/plate to minimize cross contamination risk) at 3.5⨯105 cells/well one day prior to infecting. Vero E6-TMPRSS2suggested: NoneSerial ten-fold dilutions of clarified viral supernatants were used to inoculate Vero E6 cell monolayers (60,000 cells/well seeded one day prior) in 96□well white polystyrene microplates (Thermo Fisher, Cat. #07-200-628). Vero E6suggested: NoneSoftware and Algorithms Sentences Resources Each 30 µl PCR reaction contained 5 µl of cDNA, 4.7 µl water, 14.3 µl NoROX (Qiagen, Cat. #204745), 0.7 µl Hi Rox (Qiagen, Cat. #204545), 2.5 µl each of 10 uM Tag-F and E_Sarbecco-R primers, and 0.3 µl of 10 µM E_Sarbecco-P probe. Cat.suggested: NoneConcentration of the resulting RNA was determined first by NanoDrop spectrophotometer of two high-concentration dilutions (approximately 1 µg/µl and 100 ng/µl) measured in duplicate followed by a dilution in PBS to an approximate concentration of 2×1011 copies/mL, and then by reverse transcription droplet digital PCR (RT-ddPCR) system (Bio-Rad, Hercules, CA, USA) of two low-concentration dilutions (approximately 100 and 10 copies/µl) measured in duplicate with the Mills-sgE primer/probe set. NanoDropsuggested: NoneResults from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:There are a number of limitations to the work presented here. Most importantly, it is not known whether viral isolation is a perfect laboratory correlate for viral infectivity and transmission in humans, which can vary significantly by time, distance, anatomy or mask wearing, and host immune status. We did not specifically convert CT to viral load given the multiple loci and tests investigated and the presence of mixed genomic and subgenomic transcripts for certain qRT-PCR sets. CT values are strongly assay-and instrument-dependent, and so other labs would need to validate the sensitivity of these primers against independent standard curves in order to calibrate assay performance before putting either the WHO-E CT limit or the Mills-sgE assay into use in their own labs. Finally, this work raises several important questions regarding the basic virology of SARS-CoV-2. The variation in viral titers generated from samples harvested at similar levels of CPE is intriguing, especially as we observed relatively little variation in RNA levels seen in these same samples (Supplemental Table S2). Potential variations in the ratio of infectious virus titers to RNA levels has important implications, as current dogma generally assumes a constant relationship between total RNA and levels of infectious virus in clinical samples. Higher levels of viral RNA correlate with poorer clinical outcomes for instance [38], but in general it has been difficult if not impossible to routinely measure infe...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
