[Info-vax] MIPS vs. VUPS

[Paul Sture]

On 2015-07-27, already5chosen at yahoo.com <already5chosen at yahoo.com> wrote:
> On Monday, July 27, 2015 at 12:04:45 PM UTC+3, Paul Sture wrote:
>> 
>> Something else I've just discovered on this box is that there appears
>> to be a widespread assumption that "compression is good".  The
>> specific example here is a bit of code for migrating virtual machines
>> from one host to another, which employs gzip.
>> 
>> On the Microserver this migration seemed to be taking an excessive
>> amount of time so I timed it with and without gzip on a set of files
>> which total 83GB
>> 
>> With gzip: 2.5 hours
>> Without gzip: 14 minutes
>> 
>> Gzipping the uncompressed files takes ~50 minutes on my Mac mini (and 24
>> minutes to decompress), which isn't a trivial operation either, 
>
> 24 min is approximately a time that it takes to read 23 GB and to
> write 83GB on Mac Mini's anemic 5400 rpm HDD. Uncompression itself
> close to free.

Good point.  Here are the detailed figures:

Mac mini, using a USB3 external drive with lots of free space:

$ time gzip *

real    52m8.677s
user    47m23.876s
sys      1m33.583s

and to uncompress:

$ time gunzip *.gz

real    24m7.784s
user     4m43.800s
sys      1m8.167s

Not totally free, but close enough.

A problem with gzip/gunzip is that they do their stuff in situ.  With
VMS Backup or tar you have the option of doing operations across
spindles to minimise disk head thrashing.

>
> I am not sure what is "HP NL40 Microserver".  May be, you meant N40L?

Correct, I meant N40L.

> Then its has a processor that is several (3?) times slower than
> non-very-old Mac Mini, which should be still good enough for
> decompression, but could be slow for compression. So, on your specific
> hardware SUSE assumption about "compression is good" happened to be
> wrong, but on more typical SUSE machines it could be correct.

Indeed.  A more typical usage of a low end server such as the N40L will
be where folks are generating compressed files on (faster) desktop
systems and merely storing the results on the server.

FWIW there's another possibility here and that's to split up files to be
compressed and run n jobs where n = the number of cores you have
available.  In the case of a single large file, gzip files can be
concatenated (see the man entries for gzip and gunzip), so it could be
split, the components compressed in separate jobs so that they get their
own core, then the results concatenated.


-- 
 If it jams - force it. If it breaks, it needed replacing anyway.