PathoFact 2.0: An Integrative Pipeline for Predicting Antimicrobial Resistance Genes, Virulence Factors, Toxins and Biosynthetic Gene Clusters in Metagenomes

  1. Júlia Ortís Sunyer
  2. Luis F. Delgado
  3. Oskar Hickl
  4. Cedric C. Laczny
  5. Patrick May
  6. Paul Wilmes

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Summary

Antimicrobial resistance genes (ARGs) and virulence factors (VFs) are central contributors to the global health crisis surrounding drug-resistant infections. PathoFact, a bioinformatics pipeline introduced in 2021, provides insights into ARGs, VFs, and bacterial toxins from metagenomic data. However, recent advancements in bioinformatics highlight the need for an updated version of PathoFact. We introduce PathoFact 2.0, an enhanced pipeline for improved ARG, VF, and toxin prediction. Key updates include an updated machine learning (ML) model for VF identification, a new ML model for toxin identification, expanded hidden Markov model profiles, and the antiSMASH 7.0 integration for predicting biosynthetic gene clusters. These upgrades make PathoFact 2.0 a more powerful, user-friendly platform for predicting microbiome-based pathogenicity and resistance, offering a crucial tool for better understanding and addressing the challenges posed by antimicrobial resistance and infectious diseases.

Availability and Implementation

PathoFact 2.0 is available for download at https://gitlab.lcsb.uni.lu/ESB/PathoFact2/ .

Article activity feed

  1. GigaScience

    AbstractSummary Antimicrobial resistance genes (ARGs) and virulence factors (VFs) are central contributors to the global health crisis surrounding drug-resistant infections. PathoFact, a bioinformatics pipeline introduced in 2021, provides insights into ARGs, VFs, and bacterial toxins from metagenomic data. However, recent advancements in bioinformatics highlight the need for an updated version of PathoFact. We introduce PathoFact 2.0, an enhanced pipeline for improved ARG, VF, and toxin prediction. Key updates include an updated machine learning (ML) model for VF identification, a new ML model for toxin identification, expanded hidden Markov model profiles, and the antiSMASH 7.0 integration for predicting biosynthetic gene clusters. These upgrades make PathoFact 2.0 a more powerful, user-friendly platform for predicting microbiome-based pathogenicity and resistance, offering a crucial tool for better understanding and addressing the challenges posed by antimicrobial resistance and infectious diseases.

    This work has been peer reviewed in *GigaScience *(see https://doi.org/10.1093/gigascience/giag062), which carries out single-anonymized peer review. These reviews are published under a CC-BY 4.0 license and were as follows:

    Reviewer 3：

    *Several methods are available to predict ARGs, VFs, Toxins, and Biosynthetic Gene Clusters. However, the authors selected only a few tools to benchmark PathoFact 2.0. I find this point lacking in the manuscript. To be useful to the scientific community, a more rigorous performance evaluation is needed. *It is not fully clear how the "false" sequences were chosen. Ideally, they should be similar to known resistance genes, but should not confer resistance. *Details of the parameters used to create the HMM models are not mentioned in the manuscript. The performance of the updated HMMs in comparison to the older version is not shown. *It would be interesting to show how updates in DeepARG, RGI, and AMRFinderPlus have improved the performance of PathoFact 2.0 over version 1.0. *I believe the non-pathogenic dataset was constructed using sequences other than those mentioned in the section "Generalities about Machine learning training set-up and 'non-pathogenic". This means that sequences that do not contain the mentioned keyword were used as the negative dataset. These sequences include housekeeping genes, which are also too distant from the ARG, VF, etc. The real test of an ML model occurs with data from the grey zone, which has properties of both negative and positive examples. The authors can benchmark the ML model using the grey-zone data to show the efficiency of the ML model.

    Based on the above-mentioned points, I recommend for major revision of the manuscript.

  2. GigaScience

    AbstractSummary Antimicrobial resistance genes (ARGs) and virulence factors (VFs) are central contributors to the global health crisis surrounding drug-resistant infections. PathoFact, a bioinformatics pipeline introduced in 2021, provides insights into ARGs, VFs, and bacterial toxins from metagenomic data. However, recent advancements in bioinformatics highlight the need for an updated version of PathoFact. We introduce PathoFact 2.0, an enhanced pipeline for improved ARG, VF, and toxin prediction. Key updates include an updated machine learning (ML) model for VF identification, a new ML model for toxin identification, expanded hidden Markov model profiles, and the antiSMASH 7.0 integration for predicting biosynthetic gene clusters. These upgrades make PathoFact 2.0 a more powerful, user-friendly platform for predicting microbiome-based pathogenicity and resistance, offering a crucial tool for better understanding and addressing the challenges posed by antimicrobial resistance and infectious diseases.

    This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giag062), which carries out single-anonymized peer review. These reviews are published under a CC-BY 4.0 license and were as follows:

    Reviewer 2：

    The authors present the pipeline PathoFact2.0, which combines external modules and machine learning algorithms in order to find genes that provoke antimicrobial resistance, virulence and toxicity. They present their work, the improvement from the previous version, as a Technical Note. For what the authors say, it is the only pipeline available with those characteristics, which makes it clearly a relevant software and article. However, I believe the article requires refinement, as well as new tests that support the authors claims.

    Refinements on the article In the abstract, ARG is used as an abbreviation for Antimicrobial Resistance, not Antimicrobial Resistance Genes. Overall, the article should be more clear in what the pipeline is made for. It mentions fungi, viral, etc… sequences (which might be found on a metagenomic sample, of course), but to my understanding, all the tools and phenotypes searched for are mostly characteristic of bacteria. While the introduction offers a good resume of the genes of interests, there are some descriptions that are not particularly accurate. "Human, animal, and environmental microbiomes harbour commensal and pathogenic microorganisms, contributing to the emergence of infectious diseases" seems to say that commensal microorganisms contribute to the emergence of infectious diseases; "ARGs are genetic elements that confer bacterial resistance to antibiotics, acquired via mutations or horizontal gene transfer." seems to say that antimicrobial resistance genes are acquired via mutation (they are not, there is a difference between resistance to antimicrobials provoked by mutations and by genes). I recommend a thorough rewriting of the 6 first paragraphs. The graphs in Figures 1 and 2 have different Y axes, which are also not shown. This is, to say it lightly, very misleading. Table 1 would be much more clear as a Figure. Table 1 cited in line 211 does not exist. A short description of "dereplication" would help users lacking that knowledge. The description of the parameters of the machine learning modules are a copy-paste of the variables used by scikit-learn (lines 248 and 268). The description of the machine learning models should be clearer and more detailed, as well as not force the reader to go to the instructions of scikit-learn to check what is the meaning of those parameters. The authors analyze the performance of the models depending on the "probability" (a term that could definitely use a better introduction) using barplots. A standard to analyze the probability of a model is a ROC curve. Improvements The claims of the authors about the machine learning models for VF and toxin prediction being more accurate than similarity models is, to my understanding, not proved in the article. If it is only compared to PathoFact, which was created with a dataset made years ago, the higher performance could easily be because of more complete datasets. A fair comparison of an improved performance should be done with the same dataset (PathoFact but with the dataset collected for PathoFact2.0). Moreover, the results only show that PathoFact2 predicts more toxins and virulence factors than PathoFact. The creation of the dataset, for training and most importantly for testing, is rather unclear and described all over the article. I recommend creating a section for it, to understand better the filtering, maybe a figure (could go in the supplementary material, if necessary), and include the amount of data in each test set. The authors seem to have put a lot of effort on the testing sets (including trying to avoid testing with the same data that the models are trained with) but it gets diluted in the article and, in consequence, the test results are difficult to evaluate. The results against VirulentHunter are impressive, outperforming a fine-tuned language model. While I do not doubt that the authors are thorough in their methods, such claims require more testing. Testing using external databases (not created by the authors, maybe the same used by VirulentHunter or other models validated experimentally such as pLM4VF) would support such claims. The comparative on different bacterial strains gives more questions than answers. Are all those VF and toxins found on E. coli experimentally validated? How much overlap is there between pathogenic and non-pathogenic E. coli? Are all of the same type?

    Overall, there is a good amount of work on this project, but the article still has a lot of unanswered questions. It is a bit unclear the strengths of PathoFact2, as well as its weaknesses (any model has). Could be its speed, could be having plenty of tools contained in a pipeline. I would also appreciate a better description of the report that PathoFact2.0 produces. If its strength is the virulence and toxin prediction, more tests must be performed (as described above). This would be very beneficial for possible users of the model. Moreover, in a more technical note, I recommend the authors to add a test sample for easy testing of the model in their repository.

  3. GigaScience

    AbstractSummary Antimicrobial resistance genes (ARGs) and virulence factors (VFs) are central contributors to the global health crisis surrounding drug-resistant infections. PathoFact, a bioinformatics pipeline introduced in 2021, provides insights into ARGs, VFs, and bacterial toxins from metagenomic data. However, recent advancements in bioinformatics highlight the need for an updated version of PathoFact. We introduce PathoFact 2.0, an enhanced pipeline for improved ARG, VF, and toxin prediction. Key updates include an updated machine learning (ML) model for VF identification, a new ML model for toxin identification, expanded hidden Markov model profiles, and the antiSMASH 7.0 integration for predicting biosynthetic gene clusters. These upgrades make PathoFact 2.0 a more powerful, user-friendly platform for predicting microbiome-based pathogenicity and resistance, offering a crucial tool for better understanding and addressing the challenges posed by antimicrobial resistance and infectious diseases.

    This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giag062), which carries out single-anonymized peer review. These reviews are published under a CC-BY 4.0 license and were as follows:

    Reviewer 1：

    The pipeline should be very useful on shotgun metagenomics data analysis. Aside the ARGs and VFs, the features on signal peptides, toxin predictions, and BGCs in particular for specialised metabolites predictions, are welcome for detailed analysis and understanding of various transmission mechanisms. I find the approaches very appealing and I think the pipeline could be welcomed by the community.

    I only have some minor observations:

    • The Methods section should be placed next to the described methods. As it is, at the end of the manuscript, under Methods chapter you can only find Datasets, so a proper formatting of the Methods is required
    • there is a 70 blank pages buffer between References and supplementary data
    • could you add some future prospects in the manuscript? How well is it going to be maintained - I noticed the update are quite old.
    • I would also add some more details to the limitations. For instance it is clear that the pipeline is installable on Linux platforms, but did you considered making it available also for Apple silicon series? More and more researchers use this technology, and it works as good as the Linux distributions. I also tried an install on a M series Apple silicon, but unfortunately, most of the tools in the pipeline lead to multiple errors related to python versions (most of which are old), missing old dependencies versions, libraries, etc.
  4. Version published to 10.1101/2024.12.09.627531 on bioRxiv