Compressor::Custom (Extra)

Backup can support any compressor you like. Besides the standard gzip and bzip2 compressors, multi-threaded versions of each are also available, like pigz and pbzip2. There’s also xz, the successor to LZMA, which supports multi-threading as of v5.1.1alpha.

To configure a custom compressor, simply use the following:

Model.new(:my_backup, 'My Backup') do

  # Archives, Databases, etc...

  compress_with Custom do |compression|
    compression.command = 'pbzip2 -p2'
    compression.extension = '.bz2'
  end
end

As long as the compressor accepts input on STDIN and writes to STDOUT, it should work.
i.e. tar -cf - my_files/ | %compression.command% > my_files.tar.%compression.extension%

Also, note that setting defaults for the Custom compressor works the same as with Gzip and Bzip2.

Compressor::Custom.defaults do |compressor|
  compressor.command = 'xz -1'
  compressor.extension = '.xz'
end

Compressor Benchmarks

The following tables provide some benchmarks for gzip, bzip2 and xz using a single CPU core, and using 2 CPU cores. For multi-threaded gzip and bzip2 statistics, pigz and pbzip2 were used. xz supports multi-threaded operation as of v5.1.1alpha.

Keep in mind these are simply provided to give you an idea of the differences between these compressors, and the different levels of compression they support. Compression/Decompression times will vary depending on your system. Memory usage should be the same. If you decide to configure a Custom compressor, you should perform some benchmarks on your own system, and be sure to read the compressor’s documentation.

The source for these tests was a 461 MB tar archive of the Linux Kernel v3.2.12 source distribution.
Entries in bold represent the default settings for the compressor.

Compression using one CPU core
 
Compression
Decompression
Command
Compressed Size
Time
Memory
Time
Memory
gzip -1
125,370 KB (72.81%)
00:15
1,228 KB
00:06
1,228 KB
gzip -2
119,653 KB (74.05%)
00:17
1,224 KB
00:06
1,228 KB
gzip -3
115,649 KB (74.91%)
00:19
1,228 KB
00:06
1,232 KB
gzip -4
107,445 KB (76.69%)
00:22
1,228 KB
00:05
1,228 KB
gzip -5
103,257 KB (77.60%)
00:29
1,228 KB
00:05
1,232 KB
gzip -6
101,612 KB (77.96%)
00:39
1,228 KB
00:05
1,228 KB
gzip -7
101,161 KB (78.06%)
00:46
1,228 KB
00:05
1,228 KB
gzip -8
100,859 KB (78.12%)
00:59
1,224 KB
00:05
1,228 KB
gzip -9
100,785 KB (78.14%)
01:06
1,232 KB
00:05
1,228 KB
 
bzip2 -1
94,924 KB (79.41%)
01:42
1,592 KB
00:27
1,228 KB
bzip2 -2
88,615 KB (80.78%)
01:42
2,384 KB
00:27
1,324 KB
bzip2 -3
85,605 KB (81.43%)
01:44
2,912 KB
00:28
1,588 KB
bzip2 -4
83,748 KB (81.83%)
01:48
3,704 KB
00:28
2,116 KB
bzip2 -5
82,480 KB (82.11%)
01:50
4,496 KB
00:29
2,384 KB
bzip2 -6
81,385 KB (82.35%)
01:52
5,288 KB
00:30
2,908 KB
bzip2 -7
80,629 KB (82.51%)
01:55
6,076 KB
00:31
3,172 KB
bzip2 -8
79,986 KB (82.65%)
01:58
6,868 KB
00:32
3,704 KB
bzip2 -9
79,466 KB (82.76%)
02:01
7,660 KB
00:33
3,964 KB
 
xz -1
85,996 KB (81.35%)
01:17
9,536 KB
00:17
1,960 KB
xz -2
81,778 KB (82.26%)
01:48
17,224 KB
00:16
2,984 KB
xz -3
79,704 KB (82.71%)
02:43
32,580 KB
00:15
5,028 KB
xz -4
75,247 KB (83.68%)
04:29
49,156 KB
00:15
5,000 KB
xz -5
69,517 KB (84.92%)
06:28
96,232 KB
00:14
9,096 KB
xz -6
68,167 KB (85.21%)
07:45
96,228 KB
00:14
9,096 KB
xz -7
66,825 KB (85.50%)
08:13
190,440 KB
00:14
17,280 KB
xz -8
66,045 KB (85.67%)
08:43
378,856 KB
00:13
33,668 KB
xz -9
65,419 KB (85.81%)
09:09
690,144 KB
00:13
66,432 KB
Compression using two CPU cores
 
Compression
Decompression
Command
Compressed Size
Time
Memory
Time
Memory
pigz -p2 -1
125,123 KB (72.86%)
00:09
2,596 KB
00:04
1,144 KB
pigz -p2 -2
119,352 KB (74.11%)
00:10
2,428 KB
00:04
1,148 KB
pigz -p2 -3
115,216 KB (75.01%)
00:12
2,548 KB
00:04
1,148 KB
pigz -p2 -4
107,485 KB (76.68%)
00:14
2,536 KB
00:04
1,148 KB
pigz -p2 -5
103,282 KB (77.60%)
00:18
2,560 KB
00:04
1,144 KB
pigz -p2 -6
101,631 KB (77.95%)
00:23
2,612 KB
00:04
1,148 KB
pigz -p2 -7
101,177 KB (78.05%)
00:26
2,584 KB
00:04
1,148 KB
pigz -p2 -8
100,872 KB (78.12%)
00:33
2,568 KB
00:04
1,152 KB
pigz -p2 -9
100,810 KB (78.13%)
00:37
2,572 KB
00:04
1,144 KB
 
pbzip2 -p2 -1
95,335 KB (79.32%)
00:51
8,084 KB
00:14
7,116 KB
pbzip2 -p2 -2
89,381 KB (80.61%)
00:51
10,028 KB
00:14
7,908 KB
pbzip2 -p2 -3
86,056 KB (81.33%)
00:52
11,412 KB
00:15
8,360 KB
pbzip2 -p2 -4
84,725 KB (81.62%)
00:54
12,008 KB
00:15
9,212 KB
pbzip2 -p2 -5
83,373 KB (81.91%)
00:55
14,152 KB
00:15
9,588 KB
pbzip2 -p2 -6
83,024 KB (81.99%)
00:56
15,120 KB
00:16
10,712 KB
pbzip2 -p2 -7
82,332 KB (82.14%)
00:59
17,860 KB
00:17
11,576 KB
pbzip2 -p2 -8
81,041 KB (82.42%)
01:01
17,928 KB
00:18
11,312 KB
pbzip2 -p2 -9
79,651 KB (82.72%)
01:02
19,076 KB
00:18
11,808 KB
 
xz -T2 -1
86,964 KB (81.14%)
00:37
38,288 KB
00:17
1,928 KB
xz -T2 -2
82,603 KB (82.08%)
00:55
62,724 KB
00:16
2,948 KB
xz -T2 -3
80,236 KB (82.60%)
01:26
119,480 KB
00:16
4,996 KB
xz -T2 -4
75,941 KB (83.53%)
02:14
148,776 KB
00:15
4,992 KB
xz -T2 -5
70,214 KB (84.77%)
03:14
274,728 KB
00:14
9,088 KB
xz -T2 -6
68,894 KB (85.06%)
03:54
274,748 KB
00:14
9,092 KB
xz -T2 -7
67,374 KB (85.39%)
04:27
526,624 KB
00:14
17,284 KB
xz -T2 -8
66,406 KB (85.60%)
04:56
1,028,992 KB
00:13
33,668 KB
xz -T2 -9
65,708 KB (85.75%)
05:12
1,845,168 KB
00:13
66,432 KB