[Info-vax] Some I/O results

[Mark Berryman]

I have a Mac with ~24TB of data (consisting of fairly large files) that 
needs to be rolled out to tape.  Unfortunately, my tape drives are fibre 
channel (SAN-based) and the only systems that currently have access to 
them are my VMS systems.

No problem, says I.  I'll just create a 24TB volume set, copy the data 
up, and roll it out to tape using VMS backup.

Aside:
The maximum volume size currently supported by VMS is closer to 2TiB 
than 2TB.  If we assume 2TB = 2000GB then 2TiB is somewhere between 2199 
and 2200GB.  Creating a volume at 2199GB caused VMS to complain that it 
was too big and some space would be unused.  Same at 2190GB.  I 
eventually settled on 12 volumes of 2181GB each.

Now, how to get the data onto my VMS cluster?  The Alpha and Integrity 
systems only have 1G Ethernet interfaces but the x86 system, as an ESXi 
guest, has both 1G and 10G interfaces.

SCP does not send file size information so the destination file is 
constantly extended until the copy completes.  However, I discovered 
that if I pre-allocate the file and specify the version number of the 
destination file, SCP would the equivalent of a copy/overlay.  With a 
one line change to the source code, I was able to do the same with 
Hunter Goatley's FTP server.

So, with SCP, the best I could manage was a copy around 5 MB/sec.  On 
the other hand, a Mac to Mac copy - over the same 1G LAN - achieved over 
90 MB/sec, which is close to the maximum possible.

With FTP, I could achieve around 14-15 MB/sec.

My last option was NFS.  It managed better than 20 MB/sec.  It was also 
much easier since I could NFS mount the source and then just use a copy 
command to get the data and copy pre-allocates the destination file on 
its own.

Now here is the part that was surprising to me.  As mentioned, the Alpha 
and Integrity systems were copying over a 1G Ethernet path whereas the 
x86 system was using a 10G path (the Mac has both 1G and 10G 
interfaces).  Using the exact same source and exact same I/O sizes, the 
Alpha averaged 335 I/Os per second to its target disk, the Integrity 
system managed just over 400 I/Os per second, but the x86 system 
couldn't do any better than 279 I/Os per second - even on a link that 
was 10 times faster.  (It also has a faster HBA than the other systems).

I did check kernel mode usage and there wasn't much difference between 
the 3 systems.  It simply appears that I/O on x86 is somewhat slower 
than I would have expected.  I'll check again to see if there is any 
difference once the system is built with optimizing compilers.

Mark Berryman

P.S. If anyone was wondering, it took a week to copy all of the data up 
and over 2 days to roll it out to tape.  If I ever have to restore it, 
it will still take a *lot* less time than originally creating the data did.

I really hope support for SAN-based tape drives is coming soon to x86.