Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences

  1. Kirill Kryukov
  2. Mahoko Takahashi Ueda
  3. So Nakagawa
  4. Tadashi Imanishi

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Background

Nearly all molecular sequence databases currently use gzip for data compression. Ongoing rapid accumulation of stored data calls for a more efficient compression tool. Although numerous compressors exist, both specialized and general-purpose, choosing one of them was difficult because no comprehensive analysis of their comparative advantages for sequence compression was available.

Findings

We systematically benchmarked 430 settings of 48 compressors (including 29 specialized sequence compressors and 19 general-purpose compressors) on representative FASTA-formatted datasets of DNA, RNA, and protein sequences. Each compressor was evaluated on 17 performance measures, including compression strength, as well as time and memory required for compression and decompression. We used 27 test datasets including individual genomes of various sizes, DNA and RNA datasets, and standard protein datasets. We summarized the results as the Sequence Compression Benchmark database (SCB database, http://kirr.dyndns.org/sequence-compression-benchmark/), which allows custom visualizations to be built for selected subsets of benchmark results.

Conclusion

We found that modern compressors offer a large improvement in compactness and speed compared to gzip. Our benchmark allows compressors and their settings to be compared using a variety of performance measures, offering the opportunity to select the optimal compressor on the basis of the data type and usage scenario specific to a particular application.

Article activity feed

  1. GigaScience

    Now published in GigaScience doi: 10.1093/gigascience/giaa072

    Kirill Kryukov Department of Molecular Life Science, Tokai University School of Medicine, Isehara, Kanagawa 259-1193, JapanFind this author on Google ScholarFind this author on PubMedSearch for this author on this siteFor correspondence: kkryukov@gmail.comMahoko Takahashi Ueda Department of Molecular Life Science, Tokai University School of Medicine, Isehara, Kanagawa 259-1193, JapanFind this author on Google ScholarFind this author on PubMedSearch for this author on this siteSo Nakagawa Department of Molecular Life Science, Tokai University School of Medicine, Isehara, Kanagawa 259-1193, JapanFind this author on Google ScholarFind this author on PubMedSearch for this author on this siteTadashi Imanishi Department of Molecular Life Science, Tokai University School of Medicine, Isehara, Kanagawa 259-1193, JapanFind this author on Google ScholarFind this author on PubMedSearch for this author on this site

    A version of this preprint has been published in the Open Access journal GigaScience (see paper https://doi.org/10.1093/gigascience/giaa072 ), where the paper and peer reviews are published openly under a CC-BY 4.0 license.

    These peer reviews were as follows:

    Reviewer 1: http://dx.doi.org/10.5524/REVIEW.102308 Reviewer 2: http://dx.doi.org/10.5524/REVIEW.102307

  2. Version published to 10.1093/gigascience/giaa072
  3. Version published to 10.1101/642553v2 on bioRxiv
  4. Version published to 10.1101/642553v1 on bioRxiv