[Info-vax] MIPS vs. VUPS

[Paul Sture]

On 2015-08-06, terry+googleblog at tmk.com <terry+googleblog at tmk.com> wrote:
> On Wednesday, August 5, 2015 at 11:39:10 AM UTC-4, Paul Sture wrote:
>> > If you're not tied to the [PK]ZIP format, pbzip2 is a multithreaded
>> > implementation of the bzip2 compression format:
>> > http://compression.ca/pbzip2
>> 
>> Thanks, I'll have a look.
>> 
>> I'm unsure for the original task but bzip2 is a supported alternative
>> for other stuff in this area.
>
> I don't think it has a VMS port, but that could probably be handled.
> What is more important is how many cores are available, which is
> presumably a lesser issue with newer Itaniums (Itanii?) and will not
> be an issue on x86.

On my first test of pbzip2 on a Mac, it reported that it was using 4
cores as a default.  Activity Monitor showed it going straight to 297%
CPU usage.  That settled down to around 200% once it got going, which
is probably some combination of disk speed and other stuff running
(a Mac running OS X has quite a lot going on in the background).

> I'd also check to see how saturated the disk I/O subsystem gets from a
> single-threaded bzip2 - if you don't have excess disk performance,
> parallelizing the compression phase will be less of a win.

Possibly why CPU usage dropped to 200%.

Glen suggestion of using redirection works nicely.  I haven't timed it
yet, but for the above test I redirected output to another disk and
was rewarded by a pleasant lack of disk rattling.

> I regularly use pbzip2 for files > 250GB, but I have 24 fast x86-64
> cores to throw at it and 4GB/sec disk I/O available.

Want one. ;-)

-- 
 If it jams - force it. If it breaks, it needed replacing anyway.