linux下多线程解压工具pigz

linux下有时单线程解压缩太慢了，今天测试一下Pigz吧，官方给出的是igz默认用当前逻辑cpu个数来并发压缩，无法检测个数的话，则默认并发8个线程，也可以使用-p指定线程数。需要注意的是其CPU使用比较高。

官网：http://zlib.net/pigz

通过centos7测试一下

安装：

yum install pigz

我们看看该命令参数，如下：

[cpbmaker@cpbbuilder03v ~]$ pigz --help
Usage: pigz [options] [files ...]
 will compress files in place, adding the suffix '.gz'. If no files are
 specified, stdin will be compressed to stdout. pigz does what gzip does,
 but spreads the work over multiple processors and cores when compressing.

Options:
 -0 to -9, -11 Compression level (11 is much slower, a few % better)
 --fast, --best Compression levels 1 and 9 respectively
 -b, --blocksize mmm Set compression block size to mmmK (default 128K)
 -c, --stdout Write all processed output to stdout (won't delete)
 -d, --decompress Decompress the compressed input
 -f, --force Force overwrite, compress .gz, links, and to terminal
 -F --first Do iterations first, before block split for -11
 -h, --help Display a help screen and quit
 -i, --independent Compress blocks independently for damage recovery
 -I, --iterations n Number of iterations for -11 optimization
 -k, --keep Do not delete original file after processing
 -K, --zip Compress to PKWare zip (.zip) single entry format
 -l, --list List the contents of the compressed input
 -L, --license Display the pigz license and quit
 -M, --maxsplits n Maximum number of split blocks for -11
 -n, --no-name Do not store or restore file name in/from header
 -N, --name Store/restore file name and mod time in/from header
 -O --oneblock Do not split into smaller blocks for -11
 -p, --processes n Allow up to n compression threads (default is the
 number of online processors, or 8 if unknown)
 -q, --quiet Print no messages, even on error
 -r, --recursive Process the contents of all subdirectories
 -R, --rsyncable Input-determined block locations for rsync
 -S, --suffix .sss Use suffix .sss instead of .gz (for compression)
 -t, --test Test the integrity of the compressed input
 -T, --no-time Do not store or restore mod time in/from header
 -v, --verbose Provide more verbose output
 -V --version Show the version of pigz
 -z, --zlib Compress to zlib (.zz) instead of gzip format
 -- All arguments after "--" are treated as files

随便找个目录，如下：

[cpbmaker@cpbbuilder03v ~/root]$ du -sh verified
2.1G verified

压缩对比，暂时不考虑耗费硬件资源情况：

使用gzip压缩

[cpbmaker@cpbbuilder03v ~/root]$ time tar -zcvf verified.tgz verified/
real 2m24.327s
user 2m6.630s
sys 0m3.835s

使用pigz压缩（4个线程）

[cpbmaker@cpbbuilder03v ~/root]$ time tar -cf - verified | pigz -p 4 > verified.tgz

real 0m39.514s
user 2m7.634s
sys 0m1.901s

每次压缩大小都是1.1G

使用gzip解压

[cpbmaker@cpbbuilder03v ~/root]$ time tar xf verified.tgz

real 0m48.401s
user 0m21.082s
sys 0m4.775s

使用pigz解压

[cpbmaker@cpbbuilder03v ~/root]$ time pigz -p 4 -d verified.tgz

real 0m41.892s
user 0m14.187s
sys 0m4.678s

综上所述，pigz虽然耗费了cpu,但是确实解压缩快多了