MultiSero: An Open-Source Multiplex-ELISA Platform for Measuring Antibody Responses to Infection

  1. Janie R. Byrum
  2. Eric Waltari
  3. Owen Janson
  4. Syuan-Ming Guo
  5. Jenny Folkesson
  6. Bryant B. Chhun
  7. Joanna Vinden
  8. Ivan E. Ivanov
  9. Marcus L. Forst
  10. Hongquan Li
  11. Adam G. Larson
  12. Lena Blackmon
  13. Ziwen Liu
  14. Wesley Wu
  15. Vida Ahyong
  16. Cristina M. Tato
  17. Krista M. McCutcheon
  18. Rebecca Hoh
  19. J. Daniel Kelly
  20. Jeffrey N. Martin
  21. Michael J. Peluso
  22. Timothy J. Henrich
  23. Steven G. Deeks
  24. Manu Prakash
  25. Bryan Greenhouse
  26. Shalin B. Mehta
  27. John E. Pak

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

A multiplexed enzyme-linked immunosorbent assay (ELISA) that simultaneously measures antibody binding to multiple antigens can extend the impact of serosurveillance studies, particularly if the assay approaches the simplicity, robustness, and accuracy of a conventional single-antigen ELISA. Here, we report on the development of multiSero, an open-source multiplex ELISA platform for measuring antibody responses to viral infection. Our assay consists of three parts: (1) an ELISA against an array of proteins in a 96-well format; (2) automated imaging of each well of the ELISA array using an open-source plate reader; and (3) automated measurement of optical densities for each protein within the array using an open-source analysis pipeline. We validated the platform by comparing antibody binding to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) antigens in 217 human sera samples, showing high sensitivity (0.978), specificity (0.977), positive predictive value (0.978), and negative predictive value (0.977) for classifying seropositivity, a high correlation of multiSero determined antibody titers with commercially available SARS-CoV-2 antibody tests, and antigen-specific changes in antibody titer dynamics upon vaccination. The open-source format and accessibility of our multiSero platform can contribute to the adoption of multiplexed ELISA arrays for serosurveillance studies, for SARS-CoV-2 and other pathogens of significance.

Article activity feed

  1. Version published to 10.3390/pathogens12050671
  2. Review Commons

    Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

    Learn more at Review Commons

    Reply to the reviewers

    We thank the reviewer for their input. Our response to their comments is in the attached preliminary revision plan.

  3. Review Commons

    Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

    Learn more at Review Commons

    Referee #2

    Evidence, reproducibility and clarity

    Summary:

    • Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Please place your comments about significance in section 2.

    This manuscript provides a detailed and very clear description of multiSero, which is an open source multiplex-ELISA platform for analyzing antibody responses to SARS-CoV-2 infection. This tool is a very promising step towards fully open-source multiplex testing. Using terrific visualizations the different steps involved in measuring the antibody levels is carefully explained. It starts with a clear explanation of the principle of printed antigen arrays, the usage of developed and opensource software Pysero to analyse the colorimetric signal of each spot associated with a different antigen. The colorimetric signal was read using both a commercial reader and an inexpensive, open plate reader. The comparison between the two proved that the open plate reader is as good as the commercial reader is.

    Major comments:

    • Are the key conclusions convincing? • Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? • Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. • Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. • Are the data and the methods presented in such a way that they can be reproduced? • Are the experiments adequately replicated and statistical analysis adequate?

    The authors provide a new method to measure antibody levels. A comparison with an exisisting ELISA for anti-spike IgG would be worthwhile.

    A gradient boosting tree was used to combine the signal from multiple antigens. However, this did not lead to any notable improvements in classification performance. This result could be due to a property of the data or the algorithm. Two things that would be very useful here would be to plot the data (e.g. anti-Spike vs anti-N) and use a much simpler algorithm such as a logistic regression.

    The performance of the tool is based on one positive and one negative pool. And as the the authors mention, antibody levels are highly dependent on severity and time since infection. The performance of the classifier therefore strongly depends on the characteristics of the positive pool. It would improve the manuscript by providing additional information, if possible. If not, I think this should be mentioned as a short-coming in the discussion. Possibly, having serum panel with more asymptomatic infections or longer time since infection, would result in a poorer performance from the classifier.

    Related to the point above is what is written in line 248-249. The direction of the performance of the tool with additional samples depends on the characteristics (time since infection, age, severity) of the currently used samples and the samples to be added. The assumption that the performance can only increase is in my opinion not correct.

    The authors compared three normalization methods to circumvent using a standard curve. The normalization of ODs by the mean of anti-IgG Fc ODs is most promising as shown in Fig. S5. A comparison between this normalization method and using a standard curve is not given. It would be worthwile to look at the distribution of a serum panel from different plates, in relative antibody units as well as normalized ODs. Is the captured antibody distribution by normalized ODs as good as relative antibody concentrations derived from the standard dilution.

    In the abstract, the reader is told that the multiSero tool could be used with up to 48 antigens. I assume that at this number of antigens, the use of duplicate/triplicate antigens is not possible anymore? Also, the layout and spacing of the antigen array with more antigens would introduce more experimental artificats like comets and debris ?

    In FigS3, and line 146/147 the authors state that they find the that the presence of comets odes not cause observable bias or variance. This strikes me as rather subjective, and my impression of FigS3 B3 is that there is some bias due to comets?

    Minor comments:

    • Specific experimental issues that are easily addressable. • Are prior studies referenced appropriately? • Are the text and figures clear and accurate? • Do you have suggestions that would help the authors improve the presentation of their data and conclusions?

    Important reason for developing the multiSero tool according to the authors is the deployment of high content, multiplex serology platforms across the world and this paper makes a huge step towards this goal. Main hurlde of implementing the multisero tool in low-resource settings is its dependency on printed antigen arrays, which are produced by machines costing around 100,000-300,000 $ as mentioned by the authors. The authors also realize this and acknowledge this bottleneck in the discussion. I think it would possibly be good to elaborate a little further on why this is a limitation. Less freedom with the user what they want to test because dependent on producer of printed 96 well-plates?

    In line 47, I suppose the word are is missing.

    The overall language use is very clear. An improvement in my opinion would be to replace words such as cognate (line 46) and « in lieu of » (line 227) by easier alternatives, such as associated and instead of.

    Comets and debris are first mentioned in line 129/130 but require more explanation. An explanation of what is meant with comets only became clear to me after reading the discussion. I would use the explanation mentioned in line 250/250 right after the first time mentioning comets. What debris means, remains unclear to me.

    In line 174, I suppose that the word points should be line.

    Pysero sometimes starts with a capital P, sometimes with a lower case p, see for example line 108 and 109.

    Authors find using a standard curve as labor-intensive (line 190), I find this too strongly put.

    In line 281-283 the authors mention they are unaware of examples of classifiers distinguishing positive from negative samples based on more than antigen. Examples could be the classification of cholera using 2-6 antigens by : Azman et al, 2019 in Sci. Transl. Med.

    In Fig S1 the Nauttilus plate reader is shown. The costs of this reader are estimated to be less than 1500$. These are the costs without the motorized

    Significance

    With this manuscript, the authors show that multiplex serology platforms can become more accessible to low and medium income countries due to their development of a new open source tool. This means that multiplex serology seems to be becoming more accessible in low-resource settings. Next step is to use this multiSero tool in a low-source setting.

    Specific audience potentially interested are computational biologists involved in the analysis, visualization, and interpretation of the results of techniques and microbiologists quantitating and measuring antibodies. A broader audience that could be interested are infectious disease epidemiologists, especially those that are involved in serosurveillance and are keen to pick up new methods to potentially improve epidemiologal descriptions of immunity to several infections in low-and medium-resource settings.

    My field of expertise is limited to field-epidemiology and sero-epidemiology. Techniques such as the detection of spots and registering grids with multiSero are outside of my expertise. The construction of the Nautilus reader is new to me, and therefore hard to assess how easy it would be set up such a system in low-resource settings. I also feel my expertise regarding the choice of classifiers is limited, as I have not used gradient boosting before. Further, I am not an expert in the field of new developments in multipex assays and therefore not up-to-date with the latest literature in this field.

  4. Review Commons

    Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

    Learn more at Review Commons

    Referee #1

    Evidence, reproducibility and clarity

    Summary:

    There is a need for multiplex serological tests，and ELISA is the most applicable platform. However, the current ELISA based multiplex serological tests are heavily dependent on expensive and sophisticated instruments and softwares, and this hinders the wide application. To address this challenge, by incorporating open-resourced instruments, developing new analysis software, the authors proposed an integrated platform for multiplex serological test. To test the platform, SARS-CoV-2 was included as the example. Overall, this study is more technical oriented. The major contents are the establishment and optimization of the platform. The aim is focused and clear, the design of the experiments are comprehensive. The conclusions could be supported by the data.

    Major comments:

    • Are the key conclusions convincing? Yes
    • Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? No
    • Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. No
    • Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. N/A
    • Are the data and the methods presented in such a way that they can be reproduced? Yes
    • Are the experiments adequately replicated and statistical analysis adequate? Yes

    Minor comments:

    • Specific experimental issues that are easily addressable.
    • Are prior studies referenced appropriately? Yes
    • Are the text and figures clear and accurate? Yes
    • Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No

    Significance

    • Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

    This study is more technical centered. The major contribution is the development of an ELISA-based platform for multiplex serological test. The authors intended to make their platform applicable at resource limited regions. However, the problem here is the current platform is still too complicate for wide application in real world. For a platform which may could be widely applied, especially at poor regions, it needs to meet several key features: 1. Low cost; 2. Standardized; 3. Simple (reduce operation to as few as possible). The major focus of this study is the first feature, and the other two features were bared touched. But, even "low cost" is still valuable and worth publication. The reviewer suggest the author to modify the manuscript to better reflect the fact.

    • Place the work in the context of the existing literature (provide references, where appropriate).

    The existing literatures were well referenced.

    • State what audience might be interested in and influenced by the reported findings.

    Researchers who are interested in assay development.

    • Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.

    Protein microarray technology. Assay development. SARS-CoV-2 antibody response analysis.

    The reviewer is not familiar with the software part.

    Other specific points:

    1. The authors mentioned that the multiplex serological test could be applied to differentiate infection and vaccination, in the case of SARS-CoV-2, how could this be possible if there is no specific biomarker?
    2. Have the authors also tested IgM?
    3. To simplify the normalization, the authors have tried several strategies, however, none works well. The results need to be further explained. Is there any other strategy could be attempted?
    4. What's the definition of the "background"?
    5. What's the rationale to select the two concentrations? Will more concentrations be better?
    6. The authors stated "open source analysis tools can be adapted for multiplexed detection of pathogens by printing pathogen-specific antibodies, instead of antigens". This is true, however, highly specific antibodies are required.
  5. ScreenIT

    SciScore for 10.1101/2021.05.07.21249238: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Recombinant DNA
    SentencesResources
    A codon-optimized His tagged SARS-CoV-2 Nucleocapsid (N) gene (43) was synthesized and cloned into a pET-28 vector by Twist Bioscience (San Francisco, CA, USA).
    pET-28
    suggested: RRID:Addgene_141289)

    Results from OddPub: Thank you for sharing your code and data.

    Results from LimitationRecognizer: An explicit section about the limitations of the techniques employed in this study was not found. We encourage authors to address study limitations.

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04362150Active, not recruitingLong-term Impact of Infection With Novel Coronavirus (COVID-…

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  6. Version published to 10.1101/2021.05.07.21249238v1 on medRxiv