Skip to content

Compression

Compression algorithms

  • [brotli] ★ — Generally better compression than gzip with comparable runtime.
  • bzip2 — Freely available, patent free, high-quality data compressor.
  • FastLZ — Small & portable byte-aligned LZ77 compression.
  • gzip — Single-file/stream lossless data compression utility.
  • libarchive ★ — Provides a common interface to different compression algorithms. Includes the bsdcat, bsdcpio, and bsdtar binaries.
  • lrzip — Long-range zip. Your choice of compression (lzma/bzip2/gzip). Best if you want to compress very large text files. Not that much development and lingering bugs.
  • lz4 ★ — Probably the best algorithm for fast compression. Faster than gzip. Default for ZFS.
  • lzo — "Offers pretty fast compression and extremely fast decompression."
  • lzturbo
  • p7zip — Best for indexable files. Indexed: can extract single files. CLI is cumbersome on Linux. Use -ms=off to disable solid compression (compressing multiple files together). This allows files to be extracted on at a time. [LZMA2, Indexed]
  • pixz — Pixz (pronounced pixie) is a parallel, indexing version of xz. Indexed: can extract single files. Better CLI than 7z. Got a fatal error last time I tried. [LZMA2, Indexed]
  • QuickLZ — QuickLZ is the world's fastest compression library, reaching 308 Mbyte/s per core.
  • snappy — Default in Apache Parquet files. Supposed to be faster than gzip at the expense of lower compression ratios.
  • squashfs — Compressed file system. Can be accessed in userspace using squashfuse. [gzip, lzma, lzo, xz, Indexed]
  • [xz] / [lzma] — Good compression, but slow. Slowest / best compression ratio. Faster decompression than bzip2.
  • zstd ★ — Probably the best algorithm for good compression. Compression can be a bit or a lot slower than gzip, depending on compression level. Decompression always reasonably fast. Developed at Facebook.

See also:

  • http://askubuntu.com/a/258228/229420
  • https://www.digitalocean.com/community/tutorials/an-introduction-to-file-compression-tools-on-linux-servers
  • http://unix.stackexchange.com/a/78273/78327
  • http://stackoverflow.com/a/8060136/2063031