A simple, sensitive and quantitative FACS-based test for SARS-CoV-2 serology in humans and animals

  1. Agnès Maurel Ribes
  2. Pierre Bessière
  3. Jean Charles Guéry
  4. Eloïse Joly Featherstone
  5. Timothée Bruel
  6. Remy Robinot
  7. Olivier Schwartz
  8. Romain Volmer
  9. Florence Abravanel
  10. Jacques Izopet
  11. Etienne Joly

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Serological tests are important for understanding the physiopathology and following the evolution of the Covid-19 pandemic. Assays based on flow cytometry (FACS) of tissue culture cells expressing the spike (S) protein of SARS-CoV-2 have repeatedly proven to perform slightly better than the plate-based assays ELISA and CLIA (chemiluminescent immuno-assay), and markedly better than lateral flow immuno-assays (LFIA).

Here, we describe an optimized and very simple FACS assay based on staining a mix of two Jurkat cell lines, expressing either high levels of the S protein (Jurkat-S) or a fluorescent protein (Jurkat-R expressing m-Cherry, or Jurkat-G, expressing GFP, which serve as an internal negative control). We show that the Jurkat-S&R-flow test has a much broader dynamic range than a commercial ELISA test and performs at least as well in terms of sensitivity and specificity. Also, it is more sensitive and quantitative than the hemagglutination-based test HAT, which we described recently. The Jurkat-flow test requires only a few microliters of blood; thus, it can be used to quantify various Ig isotypes in capillary blood collected from a finger prick. It can be used also to evaluate serological responses in mice, hamsters, cats and dogs. Whilst the Jurkat-flow test is ill-suited and not intended for clinical use, it offers a very attractive solution for laboratories with access to tissue culture and flow cytometry who want to monitor serological responses in humans or in animals, and how these relate to susceptibility to infection, or re-infection, by the virus, and to protection against Covid-19.

Note

This manuscript has been refereed by Review Commons, and modified thanks to the comments and suggestions from two referees. Those comments, and our replies, are provided at the end of the manuscript’s pdf, and can also be accessed by clicking on the box with a little green number found just above the “Abstract “ tab in the medRXiv window.

Article activity feed

  1. Version published to 10.1101/2021.09.06.21262027v3 on medRxiv
  2. Review Commons

    Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

    Learn more at Review Commons

    Reply to the reviewers

    We are very grateful to the two referees for their constructive comments and suggestions which have helped improve the quality of our manuscript.

    *------------------------------------------------------------------------------ ** __Reviewer #1 __(Evidence, reproducibility and clarity (Required)):

    Ribes et al developed a FACS-based serological assay to detect antibodies against the SARS-CoV-2 spike protein in various hosts. The authors described an assay that is more sensitive and quantitative, allowing the detection of anti-spike antibodies with just a few ul of blood, and highlighted the potential of the assay as an alternative to commercial ELISA-based assays against SARS-CoV-2 spike protein.

    Major concerns *

    On being quantitative analysis - the authors have used 20/130 reference serum from NIBSC as an example in figure 1. How does the RSS of the described assay compare/correlate with the Ab values in WHO standards? This should be included. * Response: We thank the referee for this helpful suggestion, and have now included the information on the IgG BAU in the legend of figure 1, and alluded to the characterisation of the 20/130 by the Expert Committee on Biological Standardization (Mattiuzzo et al., 2020) on lines 410-414 in the main text of the manuscript

    On sensitivity and specificity - AUC profiles should be performed and included. *

    Response: If the Jurkat-flow test was intended for clinical use, the precise determination

    of the sensitivity and specificity of the test would indeed be absolutely essential. As was already mentioned at the end of the introduction, the Jurkat-S&R-flow test is only destined to be used by research laboratories, for research purposes. This has now also been clarified at the end of the abstract : “Whilst the Jurkat-flow test is ill-suited and not intended for clinical use ….”

    As suggested by the referee, to establish the sensitivity and specificity of a diagnostic test, it is indeed practical to use the Area Under the Receiver Operating Characteristic (ROC) curve (AUC). A ROC curve is a plot of the true positive rate (Sensitivity) in function of the false positive rate (100-Specificity) for different cut-off points of a parameter. Determining properly the sensitivity and specificity of a test thus requires large collections of samples which are known to be either certainly positive or certainly negative, which we did not have access to.

    Are there any cross-reactivity with the other spike proteins from other CoVs? If so, what is the level of cross-reactivity? *

    Response: To assess cross-reactivity with other CoVs, we would have needed either Jurkat cells expressing the spike proteins from other CoVs, or sera with known reactivity against CoVs. Since we did not have access to such cells or sera, we were not in a position to address such a question.

    While the authors have showed that the flow-based assay has a more dynamic range, there is insufficient data showing that it is "more sensitive", as stated in the abstract. The authors should reflect this in the text. *

    Response: In the abstract, we do not state that the Jurkat-S&R-flow test is more sensitive than the ELISA, but “at least as sensitive”. On the other hand, we state that it is more sensitive than the HAT test, which it clearly is since there are more than a dozen samples on figure 2 that were positive with either or both ELISA and Jurkat-S&R-flow but were negative by HAT.

    Of note, we have recently described an improved protocol, called HAT-field, which significantly improves the sensitivity of HAT, albeit at the cost of decreased specificity (https://doi.org/10.1101/2022.01.14.22268980)

    Is trimer or monomer Spike expressed on the surface of the cells? *

    Response: Several studies have shown that, when the spike protein is expressed in human cells after transfection or transduction, it is in its native trimeric form at the cells’ surface and can even cause fusion with cells expressing the ACE2 receptor. This has now been clarified in the introduction section.

    While there are significant advantages of the flow-based assay, the authors should discuss the limitations of a flow-based assay as a serological assay, especially for sero-surveillance and cohort studies. For instance, HTS application is usually limited for cell-based assays. In addition, while the assay is relatively cheap, it is worth nothing that the cytometer is an expensive equipment that not all laboratories have. *

    Response: We bring the referee’s attention to the fact that those points are discussed at the end of the introduction (line 161-165) : “ Since the Jurkat-flow test calls for the use of both a flow cytometer and cells obtained by tissue culture, it is clearly not destined to be used broadly in a diagnostic context, but its simplicity, modularity, and performances both in terms of sensitivity and quantification capacities should prove very useful for research labs working on characterizing antibody responses directed against SARS-2, both in humans and animal models. “

    Minor concerns*: *

    Figure 1 - text and numbers in the FACS plots are too small. Please adjust. In addition, for some of the FACS plots shown (eg. neg cont and serum 20/130), the population is right at the axis. Please pan the x-axis to allow better visualisation.

    1. Figure 3A - please label axis.
    2. Figure S2 - please label axis.
    3. In general, please check through all figures for axis labels and also adjust the front size. For most, the text is too small. * Response: Sizes of numbers and text increased, and axis labels added in all figures*

    Reviewer #1 (Significance (Required)):

    As already discussed by the authors, there have already been quite a number of studies that have demonstrated the advantages of a flow-based assay for serological analysis for SARS-CoV-2. However, Ribes et al showed a new way to separate out alloreactivity from specific staining, which is important in reducing false positivity in serological assay. As more and more people receive their vaccination, there is a significant interest in immune-monitoring following vaccination. Given the more dynamic range of the flow-based assay, this is one good way to monitor antibody response. *

    Expertise*: **My research interest focuses on the study of SARS-CoV-2 antibody responses following infection or vaccination. **

    Reviewer #2* (Evidence, reproducibility and clarity (Required)): **

    In this paper, Joly and colleagues make use of a flow cytometry-based assay to measure in a reliable and sensitive manner the presence of IgG, IgA and IgM in blood samples from post-COVID human patients and also from laboratory (mouse and hamster) and domestic animals (dogs and cats). They find that the test is appropriate to detect the presence of humoral immunity in all species tested. The manuscript is clearly written and the Figures are clearly presented. The experiments with rodente deliberately infected with inactivated SARS-CoV-2 shows (Fig. 3) that the method is reliable and able to clearly discriminate positive from negative sera. Interestingly, dogs and cats were sampled from households in which the owners had been found to have passed COVID-19 by PCR. Among this cohort of house animals they find more than 90% seroconversion for dogs and slightly less than 30% of clear seroconversion in cats. We find however that the manuscript would benefit by establishing a clear cut-off value of "Specific Stain" for dogs and cats (Fig. 3). This could be implemented by including data from pre-COVID dog and cat sera or in its defect, sera from those species collected at households in which their owners were vaccinated and did not pass the infection. Another point of criticism that could be resolved is that the channels for flow cytometry in Figure 1 do not seem to be adequately compensated and there is evidence of some cross-contamination between FL1 and FL3. *

    Responses: We thank the referee for bringing our attention to the fact that we had presented the data on sera from cats and dogs in a confusing manner, which led the referee to believe that the sets of samples presented were representative of the population of animals whose owner had tested positive for Covid-19. In fact, for this experiment, which was only ever intended as a preliminary proof of concept that the test could be adapted very simply to companion animals, we used sets of sera which we knew would contain approximately 50 % of positive and 50 % negative samples because they had previously been screened by sero-neutralisation (incidentally, a manuscript by Bessière et al., describing that work on sera from 131 cats and 156 dogs, has very recently been submitted for publication). To prevent possible confusions, we have now reworded the description of this proof of concept experiment, in the legend of figure 3, the text, and the methods section.

    Regarding the question of a clear cut-off value, as when using human samples, we would suggest using a value of 40 for the instruments settings we used, corresponding to an RSS of 20 (i.e. 20 fold the value of the negative control). With such a value, it can be seen that one cat serum would be considered positive whilst showing no neutralising activity, but one dog serum which showed weak neutralising activity would be considered negative. If anything, this example highlights the difficulty in setting a precise cut off value for any biological test.

    Regarding the question of inadequate compensation between channels 1 and 3, this is due to the fact that the Cellquest software does not allow for FL1/FL3 compensation, which is explained in the figure legend (see lines 208-210). We decided to simply draw the gates as they appear on figure 1 because attempts at post-acquisition compensation using the Flowjo software did not give satisfactory results. Incidentally, no compensation is required when samples are acquired on a Fortessa flow cytometer, where mCherry can be excited by a different laser (see figure S1) or if one uses the Jurkat-S&G-flow version of the test as in figure 3D for hamster sera (using Jurkat-GFP as negative control, and secondary antibodies conjugated to Alexa 488).

    Minor points*: *

    *-Figure 1.- Please describe the y- and x-axis. Such as they are is difficult to find out. *

    Response: Done

    -It would be advisable to mention in Materials and Methods (page 22) how blood was collected from cats and dogs. *

    Response: We thank the referee for highlighting this, and have now provided the information in the relevant method section.

    -Line 856, page 22, "ad libidum" should be "ad libitum" *

    Response: We thank the referee for spotting this typo, which has been corrected

    Reviewer #2 (Significance (Required)):

    This is another step in the implementation of flow cytometry tests, instead of ELISA or CLIA serological tests based on the use of recombinant proteins, as a more sensitive and reliable method. The description of the high frequency of human-domestic animal transfer of SARS-CoV-2 will also add to the idea that it is humans who transmit the virus to those animals. *

  3. Review Commons

    Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

    Learn more at Review Commons

    Referee #2

    Evidence, reproducibility and clarity

    In this paper, Joly and colleagues make use of a flow cytometry-based assay to measure in a reliable and sensitive manner the presence of IgG, IgA and IgM in blood samples from post-COVID human patients and also from laboratory (mouse and hamster) and domestic animals (dogs and cats). They find that the test is appropriate to detect the presence of humoral immunity in all species tested.

    The manuscript is clearly written and the Figures are clearly presented. The experiments with rodente deliberately infected with inactivated SARS-CoV-2 shows (Fig. 3) that the method is reliable and able to clearly discriminate positive from negative sera. Interestingly, dogs and cats were sampled from households in which the owners had been found to have passed COVID-19 by PCR. Among this cohort of house animals they find more than 90% seroconversion for dogs and slightly less than 30% of clear seroconversion in cats.

    We find however that the manuscript would benefit by establishing a clear cut-off value of "Specific Stain" for dogs and cats (Fig. 3). This could be implemented by including data from pre-COVID dog and cat sera or in its defect, sera from those species collected at households in which their owners were vaccinated and did not pass the infection. Another point of criticism that could be resolved is that the channels for flow cytometry in Figure 1 do not seem to be adequately compensated and there is evidence of some cross-contamination between FL1 and FL3.

    Minor points:

    • Figure 1. Please describe the y- and x-axis. Such as they are is difficult to find out.
    • It would be advisable to mention in Materials and Methods (page 22) how blood was collected from cats and dogs.
    • Line 856, page 22, "ad libidum" should be "ad libitum"

    Significance

    This is another step in the implementation of flow cytometry tests, instead of ELISA or CLIA serological tests based on the use of recombinant proteins, as a more sensitive and reliable method. The description of the high frequency of human-domestic animal transfer of SARS-CoV-2 will also add to the idea that it is humans who transmit the virus to those animals.

  4. Review Commons

    Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

    Learn more at Review Commons

    Referee #1

    Evidence, reproducibility and clarity

    Ribes et al developed a FACS-based serological assay to detect antibodies against the SARS-CoV-2 spike protein in various hosts. The authors described an assay that is more sensitive and quantitative, allowing the detection of anti-spike antibodies with just a few ul of blood, and highlighted the potential of the assay as an alternative to commercial ELISA-based assays against SARS-CoV-2 spike protein.

    Major concerns:

    1. On being quantitative analysis - the authors have used 20/130 reference serum from NIBSC as an example in figure 1. How does the RSS of the described assay compare/correlate with the Ab values in WHO standards? This should be included.
    2. On sensitivity and specificity - AUC profiles should be performed and included.
    3. Are there any cross-reactivity with the other spike proteins from other CoVs? If so, what is the level of cross-reactivity?
    4. While the authors have showed that the flow-based assay has a more dynamic range, there is insufficient data showing that it is "more sensitive", as stated in the abstract. The authors should reflect this in the text.
    5. Is trimer or monomer Spike expressed on the surface of the cells?
    6. While there are significant advantages of the flow-based assay, the authors should discuss the limitations of a flow-based assay as a serological assay, especially for sero-surveillance and cohort studies. For instance, HTS application is usually limited for cell-based assays. In addition, while the assay is relatively cheap, it is worth nothing that the cytometer is an expensive equipment that not all laboratories have.

    Minor concerns:

    1. Figure 1 - text and numbers in the FACS plots are too small. Please adjust. In addition, for some of the FACS plots shown (eg. neg cont and serum 20/130), the population is right at the axis. Please pan the x-axis to allow better visualisation.
    2. Figure 3A - please label axis.
    3. Figure S2 - please label axis.
    4. In general, please check through all figures for axis labels and also adjust the front size. For most, the text is too small.

    Significance

    As already discussed by the authors, there have already been quite a number of studies that have demonstrated the advantages of a flow-based assay for serological analysis for SARS-CoV-2. However, Ribes et al showed a new way to separate out alloreactivity from specific staining, which is important in reducing false positivity in serological assay. As more and more people receive their vaccination, there is a significant interest in immune-monitoring following vaccination. Given the more dynamic range of the flow-based assay, this is one good way to monitor antibody response.

    My research interest focuses on the study of SARS-CoV-2 antibody responses following infection or vaccination.

  5. Version published to 10.1101/2021.09.06.21262027v2 on medRxiv
  6. ScreenIT

    SciScore for 10.1101/2021.09.06.21262027: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsField Sample Permit: Experiments on Cat and Dog sera: Serum samples were collected from cats and dogs belonging to owners who developed COVID-19-like symptoms and subsequently tested positive for SARS-CoV-2 infection by RT-qPCR.,
    IRB: Samples and data collections were conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee Sciences et Santé Animale n°115 (protocol code COVIFEL approved on 1 September 2020, registered under SSA_2020_010).
    Euthanasia Agents: Cells were then incubated for 72 h at 37 °C with 5% of CO2.
    Consent: Ethical statement: All sera from the first cohort, and whole blood samples from the second cohort, were obtained from the Toulouse hospital, where all patients give, by default, their consent for any biological material left over to be used for research purposes after all the clinical tests requested by doctors have been duly completed.
    Sex as a biological variablenot detected.
    RandomizationAll the positive samples of this cohort, as well as a set of randomly selected negatives, were submitted to a repeat of the assay, which showed excellent reproducibility.
    BlindingAfter transferring the pictures to computer files, the hemagglutination tests were scored by three independent assessors, of which two were blinded.
    Power Analysisnot detected.
    Cell Line AuthenticationContamination: The Jurkat-S and Jurkat-R cell lines were both checked for the absence of mycoplasma contamination using the HEK blue hTLR2 kit (Invivogen, Toulouse, France) FACS staining: Before experiments, cells in the cultures of both Jurkat-S and Jurkat-R cell lines were counted, and sufficient numbers harvested to have a bit more than 105 cells of each per sample to be tested.

    Table 2: Resources

    Antibodies
    SentencesResources
    Reagents: Polyclonal anti-human and anti-mouse Igs secondary antibodies, all conjugated to Alexa-488, were from Jackson laboratories, and purchased from Ozyme (France)
    anti-mouse Igs secondary
    suggested: None
    109-545-129; anti-mouse Ig-G: 115-545-003 Anti-cat IgG (F4262) and anti-dog IgG (F7884) secondary antibodies, both conjugated to FITC, were obtained from Sigma. Anti RBD monoclonal antibodies: FI3A (site 1) and FD-11A (site 3) (Huang et al. 2021); C121 (site 2) (Robbiani et al. 2020); CR3022 (site 4) (ter Meulen et al. 2006); EY6A (site 4) (Zhou et al. 2020).
    109-545-129; anti-mouse Ig-G: 115-545-003 Anti-cat IgG
    suggested: None
    anti-dog IgG
    suggested: (Sigma-Aldrich Cat# F7884, RRID:AB_259743)
    F7884
    suggested: (Sigma-Aldrich Cat# F7884, RRID:AB_259743)
    Anti RBD
    suggested: (BioLegend Cat# 944803, RRID:AB_2892509)
    All those were obtained using antibody-expression plasmids, as previously described (Townsend et al. 2021).
    antibody-expression
    suggested: None
    For specific isotyping, i.e. for staining separately with either anti-IgG, -IgA or -IgM, as well as the pan-specific anti-Ig-GAM secondary antibody, we used double the quantities of cells and of serum or plasma for the primary step, and split the samples into 4 wells after the second wash, before proceeding to the subsequent steps as for the standard protocol.
    anti-IgG
    suggested: None
    anti-Ig-GAM
    suggested: None
    Monoclonal anti-SARS-CoV CR3022 antibody (BEI Resources, NIAID, NIH) was used as a positive control.
    anti-SARS-CoV CR3022
    suggested: None
    Experimental Models: Cell Lines
    SentencesResources
    Lentiviral infectious supernatants were obtained after transient transfection of HEK cells with pLV-or pCDH-derived vectors, together with the packaging R8-2, and VSV-G plasmids.
    HEK
    suggested: None
    The Jurkat-S and Jurkat-R cell lines were both checked for the absence of mycoplasma contamination using the HEK blue hTLR2 kit (Invivogen, Toulouse, France) FACS staining: Before experiments, cells in the cultures of both Jurkat-S and Jurkat-R cell lines were counted, and sufficient numbers harvested to have a bit more than 105 cells of each per sample to be tested.
    Jurkat-R
    suggested: None
    Each sample requires 2.105 jurkat cells, which is roughly the amount obtained with 0.5 ml of standard tissue culture medium, which costs around 50 €/L, i.e. 2.5 cts/sample.
    jurkat
    suggested: None
    Experiments on mouse sera: Virus preparation and inactivation: SARS-CoV2 was grown on Vero E6 cells (ATCC) in DMEM (Dutscher) supplemented with 100 U/mL penicillin, 100 μg/ml streptomycin (Invitrogen), and 2% heat inactivated fetal bovine serum (Sigma).
    Vero E6
    suggested: None
    Sero-neutralisation assay: Serum samples and controls were heat-inactivated at 56 °C for 30 min, serially diluted in DMEM starting at 1:10, mixed with an equal volume of SARS-CoV-2 stock (previously amplified and titrated on Vero-E6 cells and diluted in DMEM to contain 2000 TCID50/ml), incubated for 2 hours at 37 °C, and 100 µL transferred to tissue-culture 96 well plates plated with 12.000 Vero-E6 cells per well the day before the assay, in DMEM complemented with 10% of heat-inactivated fetal bovine serum and 1% of penicillin-streptomycin at 37 °C with 5% of CO2 (medium was removed before adding the virus-serum dilutions).
    Vero-E6
    suggested: None
    Recombinant DNA
    SentencesResources
    pLV-EF1a-SARS-CoV-2-S-IRES-Puro was created by cloning a codon-optimized version of the SARS-CoV-2 S gene (GenBank: QHD43416.1) into the pLV-EF1a-IRES-Puro backbone (Addgene plasmid # 85132 ; http://n2t.net/addgene:85132 ; RRID:Addgene_85132) using BamHI and EcoRI sites.
    pLV-EF1a-SARS-CoV-2-S-IRES-Puro
    suggested: None
    detected: RRID:Addgene_85132)
    The lentiviral vector for the expression of mCherry was obtained by replacing the GFP sequence of GFP by that of mCherry in the pCDH-EF1α-MCS*-T2A-GFP plasmid (https://systembio.com/shop/pcdh-ef1α-mcs-t2a-gfp-cdna-single-promoter-cloning-and-expression-lentivector/
    pCDH-EF1α-MCS*-T2A-GFP
    suggested: None
    Lentiviral infectious supernatants were obtained after transient transfection of HEK cells with pLV-or pCDH-derived vectors, together with the packaging R8-2, and VSV-G plasmids.
    pCDH-derived
    suggested: None
    VSV-G
    suggested: RRID:Addgene_138479)
    The Jurkat-R cells were obtained by successive transduction with the pCDH-GFP lentiviral vector described above, then with the empty pLV lentiviral vector, followed by selection with Puromycin at 10µg/mL.
    pCDH-GFP
    suggested: None
    pLV
    suggested: None
    Software and Algorithms
    SentencesResources
    Post-acquisition analysis of all the samples was performed using the Flowjo software (version 10.7.1) Cost of the Jurkat-S&R-flow test: The cost per sample of the Jurkat-S&R-flow test lies in large part with the price of the secondary antibodies used.
    Flowjo
    suggested: (FlowJo, RRID:SCR_008520)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    As alluded to in the introduction, one of the possible caveats of using a human cell line to express the S protein is that some blood samples will contain allo-reactive antibodies directed against that cell line, possibly as a consequence of a pregnancy, or past history of receiving a blood transfusion or organ transplant (Hickey et al. 2016; Karahan et al. 2020). The third column of Figure 1 shows examples of such samples containing marked levels of alloreactive antibodies, i.e. samples for which the J-R cells show significant levels of staining compared to the same cells labelled with just the secondary antibody. Based on our results collected on more than 350 clinical samples, we evaluate that ca. 30 % of samples will contain allo-reactive Abs that will result in levels of staining of Jurkat cells that are more than five-fold that of the signal obtained for the negative control (and 3-6 % more than ten-fold). Of note, we did not notice an increased frequency of allo-reactivity in samples from women compared to men, which suggests that allo-reactivity after pregnancy is not a major cause in the origin of those allo-reactions. A difficult question with all serological tests is that of where to set the threshold beyond which the specific signals detected can confidently be considered as positive, which will be directly linked to the balance between sensitivity and specificity of the assay. Based on the analyses of various cohorts of positive and negative samples (some of whic...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  7. Version published to 10.1101/2021.09.06.21262027v1 on medRxiv