LEMMIv2: Benchmarking Framework for Metagenomic and 16S Amplicon Profilers with a Catalogue of Evaluated Tools

Sequencing has transformed microbial studies, enabling metagenomic analysis of microbial communities without the need for culturing or prior knowledge of sample composition. The essential analysis of the primary sequencing reads, however, is complex and led to the development of diverse computational strategies. The plethora of available methods and their numerous parameters poses a practical challenge for practitioners and creates a visibility barrier for developers of novel approaches. In addition to technical limitations related to user computing environment, algorithmic solutions, and their scalability, there are critical considerations regarding the reference database, i.e. the knowledge against which the data is interpreted, as well as the target of the analysis, expected sample composition, and peculiarities of read data from different sequencing platforms. To facilitate informed decision-making, we introduced the LEMMI platform for continuous benchmarking of software tools for metagenomic analyses, where developers can receive impartial benchmarks for method publication and users benefit from a standardized and benchmarked catalogue of tools. Here we present developments of LEMMI version 2, including assessments of different target scenarios, long- and short-read sequencing data, alternative taxonomies, and a standalone pipeline ( https://lemmi.ezlab.org ). In addition to LEMMI, which focuses on shotgun metagenomic profiling, we extended this approach to bacteria profiling with 16S amplicon sequencing ( https://lemmi16S.ezlab.org ).