GenPipes: an open-source framework for distributed and scalable genomic analyses

  1. Mathieu Bourgey
  2. Rola Dali
  3. Robert Eveleigh
  4. Kuang Chung Chen
  5. Louis Letourneau
  6. Joel Fillon
  7. Marc Michaud
  8. Maxime Caron
  9. Johanna Sandoval
  10. Francois Lefebvre
  11. Gary Leveque
  12. Eloi Mercier
  13. David Bujold
  14. Pascale Marquis
  15. Patrick Tran Van
  16. David Anderson de Lima Morais
  17. Julien Tremblay
  18. Xiaojian Shao
  19. Edouard Henrion
  20. Emmanuel Gonzalez
  21. Pierre-Olivier Quirion
  22. Bryan Caron
  23. Guillaume Bourque

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Background

With the decreasing cost of sequencing and the rapid developments in genomics technologies and protocols, the need for validated bioinformatics software that enables efficient large-scale data processing is growing.

Findings

Here we present GenPipes, a flexible Python-based framework that facilitates the development and deployment of multi-step workflows optimized for high-performance computing clusters and the cloud. GenPipes already implements 12 validated and scalable pipelines for various genomics applications, including RNA sequencing, chromatin immunoprecipitation sequencing, DNA sequencing, methylation sequencing, Hi-C, capture Hi-C, metagenomics, and Pacific Biosciences long-read assembly. The software is available under a GPLv3 open source license and is continuously updated to follow recent advances in genomics and bioinformatics. The framework has already been configured on several servers, and a Docker image is also available to facilitate additional installations.

Conclusions

GenPipes offers genomics researchers a simple method to analyze different types of data, customizable to their needs and resources, as well as the flexibility to create their own workflows.

Article activity feed

  1. GigaScience

    Now published in GigaScience doi: 10.1093/gigascience/giz037

    Mathieu Bourgey 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteFor correspondence: guil.bourque@mcgill.ca mathieu.bourgey@mcgill.caRola Dali 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteRobert Eveleigh 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteKuang Chung Chen 3McGill HPC Centre, McGill University, Montréal, QC, Canada.4Calcul Québec, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteLouis Letourneau 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteJoel Fillon 5Department of Human Genetics, McGill University, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteMarc Michaud 2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteMaxime Caron 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.5Department of Human Genetics, McGill University, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteJohanna Sandoval 6Beaulieu-Saucier Université de Montréal Pharmacogenomics Centre, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteFrancois Lefebvre 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteGary Leveque 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteEloi Mercier 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteDavid Bujold 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this sitePascale Marquis 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this sitePatrick Tran Van 7Department of Ecology and Evolution, University of Lausanne, Lausanne, Switzerland.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteDavid Morais 8Centre de calcul scientifique (ccs) - Université de Sherbrooke, Sherbrooke, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteJulien Tremblay 9Energy, Mining and Environment, National Research Council Canada, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteXiaojian Shao 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteEdouard Henrion 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteEmmanuel Gonzalez 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this sitePierre-Olivier Quirion 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteBryan Caron 3McGill HPC Centre, McGill University, Montréal, QC, Canada.4Calcul Québec, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteGuillaume Bourque 1Canadian Centre for Computational Genomics, Montréal, QC, Canada.2McGill University and Genome Québec Innovation Center, Montréal, QC, Canada.5Department of Human Genetics, McGill University, Montréal, QC, Canada.Find this author on Google ScholarFind this author on PubMedSearch for this author on this siteFor correspondence: guil.bourque@mcgill.ca mathieu.bourgey@mcgill.ca

    A version of this preprint has been published in the Open Access journal GigaScience (see paper https://doi.org/10.1093/gigascience/giz037 ), where the paper and peer reviews are published openly under a CC-BY 4.0 license.

    These peer reviews were as follows:

    Reviewer 1: http://dx.doi.org/10.5524/REVIEW.101755 Reviewer 2: http://dx.doi.org/10.5524/REVIEW.101756 Reviewer 3: http://dx.doi.org/10.5524/REVIEW.101759

  2. Version published to 10.1093/gigascience/giz037
  3. Version published to 10.1101/459552v1 on bioRxiv