[Info-vax] SCP2 and large files

[bill.hall at navistar.com]

On Wednesday, August 12, 2020 at 1:47:52 AM UTC-5, Jeremy Begg wrote:
> Hi!
> 
> A customer running HP OpenVMS V8.4 with TCP/IP Services V5.7-ECO5 is 
> having a problem with SCP2.
> 
> The nightly VMS backup job creates BACKUP savesets on a local disk.
> When each saveset is completed (i.e. when the BACKUP/IMAGE finishes) a 
> batch job is submitted to transfer the saveset to a Linux system using 
> SCP2.  (From there it goes to a large archiving system.)
> 
> Those transfers work fine for most of the savesets, which vary in size 
> from 9GB to 52GB.  However there's one saveset which is close to 500GB 
> for which the transfer never works.
> 
> It appears that SCP2 does the following:
> 
> 1. Connects to the remote (Linux) server and logs in.
> 2. Does a stat() on the saveset (presumably to get its exact size).
> 3. Starts transferring the file.
> 
> I think where it's going wrong is step 2.  The stat() on the 500GB 
> saveset takes a very long time (seems to be about 90 minutes), by which 
> time the Linux end has given up waiting for the transfer to begin.
> 
> Therefore two possible solutions spring to mind:
> 
> a. Can the CRTL stat() routine be told to guess the file size from the 
> RMS file header (ALQ perhaps)?  I thought there might be a DECC$ feature 
> logical to do this, but I can't find it.
> 
> b. Can SCP2 be told to enable TCP keepalive on the connection?
> 
> Thanks,
> 
> 	Jeremy Begg

As a work around to get smaller savesets have you tried backup's compression on the savesets?

I've been doing backups on one server to remote savesets on another VMS server lately (using DECnet-plus FAL, not SCP after the saveset creation).  My bottleneck is the network, not CPU or disk I/O and I've found that the reduced saveset size it really helps reduce the elapsed time.  I've seen between 50% and almost 90% compression on a saveset depending on the contents of the source.

$ backup /data_format=compress=deflate/group=0 <source disk>/image -
  remote::<save_set>/save_set

>From a recent log file:
%BACKUP-I-COMPRESS, data compressed by 74%
%BACKUP-I-COMPRESS, data compressed by 49%
%BACKUP-I-COMPRESS, data compressed by 89%