Article activity feed

  1. Evaluation Summary:

    This is an interesting report using computational tools and large amounts of prospective samples from clinical trials to identify different signatures. Using data collected early in the infection in outpatients, the authors aim to identify a set of plasma proteins that can predict a number of outcomes, including disease progression, control of viral shedding, and the onset of antibodies during COVID-19. This study adds to the understanding of the host immune response against COVID-19, as well as the potential of computational tools for the molecular taxonomy of immune responses.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

    Was this evaluation helpful?
  2. Reviewer #1 (Public Review):

    An interesting report by authoritative investigators using computational tools and large amounts of prospective samples from clinical trials to identify different signatures (eg, related to interferon signaling or to cytokine/chemokine production) associated to clinical outcomes in COVID-19. An additional value is the comparison investigation of these signatures also in the context of COVID-19 vaccination. This study adds to the understanding of the host immune response against COVID-19, as well as the potential of computational tools for the molecular taxonomy of immune responses. The next logical step is to apply this methodological approach to predict responses to treatment at the individual levels or to instruct enrolment in clinical trials or administration of targeted immunomodulatory agents.

    Was this evaluation helpful?
  3. Reviewer #2 (Public Review):

    The study of Hu et al. aims to decipher early immune signatures of severe forms of COVID-19. Using data collected early in the infection in outpatients, the authors identify a set of plasma proteins that can predict a number of outcomes, including disease progression, control of viral shedding and onset of antibodies.
    The study is interesting but I have a number of questions on the methodology and the implications of this work. I would really encourage the authors to clarify the methods so that other groups can build on it. As such a number of details need to be clarified to ensure full reproducibility.

    1. The authors take for granted that the administration of Peg-IFN lambda does not modify the course of the disease and therefore that treated and untreated individuals can be analyzed together. This is at odds with other randomized studies, that have shown antiviral and clinical effect of IFN-based therapy. In particular Peg-lambda accelerated viral decline in outpatients and prevented clinical deterioration in a study performed in a similar setting using the same dose than here (Feld et al., Lancet Resp Med 2021). Other positive results in early patients were found with IFN-beta (Monk et al, Lancet Resp Med 2020).
    2. Even if the administration of IFN in this study had no clinical or virological benefits, it could nonetheless alter the kinetics of ISG. The authors claim it is not the case, but it is difficult to assess it based on the figures shown, using PCA.
    3. I really doubt that any strong claim can be made on disease progression since only 8 patients were hospitalized in this study. In addition, it should be clarified when these patients progressed. Page 10, it is said that the median time to progression is 2 days, so in fact the data collected at day 0 and 5 are very close, or even perhaps posterior to hospitalization in some cases, making it difficult to claim that it can be used for prediction. More generally data used are up to 14 days post symptom onset, while the median time to hospitalization in these populations is roughly ~8 days. This makes it here as well difficult to really argue that the model has a "predictive" value to anticipate disease progression.
    4. If data used in the study are close to hospitalization, then this really diminishes the novelty of the findings, as many studies have already reported an association between these markers and disease severity (see also Young et al, Viral Dynamics and Immune Correlates of Coronavirus Disease 2019 (COVID-19) Severity, CID 2021).
    5. The definition of disease progression seems to differ from the original study "Overall, 17 participants had evidence of disease progression, defined as hospitalization, presentation to the emergency department, or worsening cough or shortness of breath defined as an increase in severity of two points or more on a five-point scale"? Please clarify what is your endpoint and why, if relevant, it differs from the original study.
    6. Can you clarify how viral shedding was analyzed? I am puzzled by the fact that viral load is analyzed with a different metrics than other proteins when looking at predictors of disease progression? I am also not convinced by Fig S5 which relies on AUC of viral load calculated in patients with high heterogeneity in their symptom onset. Please use the same approach for viral load than what was used for IP-10 in order to demonstrate that IP-10 is a better predictor of disease progression than viral load.
    7. Regarding prediction, it is really unclear how the model using demographics was built. It is obvious than many other factors than age and sex are highly predictive of disease progression.
    8. If you want these results to be useful for the clinical community then the model used in figure 7 should be explicitly given so that any one can use these results to build score on its own population.

    Was this evaluation helpful?
  4. SciScore for 10.1101/2021.08.27.21262687: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsIRB: The study was performed as an investigator-initiated clinical trial with the FDA (IND 419217), and approved by the Institutional Review Board of Stanford University.
    Sex as a biological variablenot detected.
    RandomizationLambda Study Design and Oversight: Data and samples were obtained from a Phase 2, single-blind, randomized placebo-controlled trial to evaluate the efficacy of Lambda in reducing the duration of viral shedding in outpatients.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Antibodies
    SentencesResources
    Following primary incubation, 25 μl of 1:5000 diluted horse radish peroxidase (HRP) conjugated anti-Human IgG secondary antibodies (Southern Biotech) were added and incubated for 1h at RT.
    anti-Human IgG
    suggested: None
    Predictive modeling: We used the protein measurements (measured by Olink assays) to predict the clinical, virological and memory T cell activity and IgG antibodies.
    IgG
    suggested: None
    Software and Algorithms
    SentencesResources
    Participant Follow Up and Sample Collection: Participants completed a daily symptom questionnaire using REDCap Cloud version 1.5.
    REDCap
    suggested: (REDCap, RRID:SCR_003445)
    Clinical Laboratory procedures: Laboratory measurements were performed by trained study personnel using point-of-care CLIA-waived devices or in the Stanford Health Care Clinical Laboratory.
    Clinical Laboratory
    suggested: None
    Oropharyngeal swabs were tested for SARS-CoV-2 in the Stanford Clinical Virology Laboratory using an emergency use authorized, laboratory-developed, RT-PCR. Centers for Disease Control and Prevention guidelines identify oropharyngeal swabs as acceptable upper respiratory specimens to test for the presence of SARS-CoV-2 RNA, and detection of SARS-CoV-2 RNA swabs using oropharyngeal swabs was analytically validated in the Stanford virology laboratory.
    Stanford Clinical Virology Laboratory
    suggested: None
    Whole blood transcriptomic data analysis: The transcript-level count data and transcript per million (TPM) data was calculated using Kallist47 (v0.46.2) and human cDNA index produced using kallisto on Ensembl v96 transcriptomes.
    Ensembl
    suggested: (Ensembl, RRID:SCR_002344)
    All samples were analyzed on an Attune NXT flow cytometer and analyzed with FlowJo X (Tree Star)
    FlowJo
    suggested: (FlowJo, RRID:SCR_008520)
    All data were normalized between the same positive and negative controls and the binding AUC were calculated using GraphPad PRISM (Version 9)
    GraphPad PRISM
    suggested: (GraphPad Prism, RRID:SCR_002798)
    We estimated the enrichment score of the major immune cell types using the xCell package23.
    xCell
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).


    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study has some limitations. First, while we identified multiple associations between early immune measures and the outcome of COVID-19 patients, we did not establish causal relationships between them. Future studies are needed to perturb key immune pathways in the early immune response and test their effect on the patient outcomes. Second, our study measured the immune response during the first 2 weeks of symptom onset in COVID-19 patients. Earlier immune responses between the initial infection and symptom onset have not been characterized. This is due to the difficulty to detect pre-symptomatic COVID-19 infection. Routine SARS-COV-2 monitoring in a select cohort will be required to acquire samples prior to and immediately after the infection in order to assess whether pre-infection signatures predict outcomes in COVID-19 patients. Third, our analysis focused on individual plasma proteins (based on olink data) and immune related Gene Ontology pathways (based on RNA-seq data). We used the Gene Ontology-based pathways to provide a high-level overview of the immune response in COVID-19 patients. Caution should be taken when interpreting the Gene Ontology pathways results, as the pathways are manually curated gene lists from literature and subject to publication bias, curation errors and over-simplification of biological processes. We encourage others to investigate the immune response of individual genes of interest using our shared RNA-seq data. Finally, we have created mac...

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04331899CompletedSingle-Blind Study of a Single Dose of Peginterferon Lambda-…


    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.


    Results from JetFighter: We did not find any issues relating to colormaps.


    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a protocol registration statement.

    Results from scite Reference Check: We found no unreliable references.


    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

    Was this evaluation helpful?