[Info-vax] VMS to VMS data copy options/performance when losing a DECnet link

[Rich Jordan]

On Friday, April 8, 2022 at 6:17:59 PM UTC-5, geze... at rlgsc.com wrote:
> On Friday, April 8, 2022 at 11:30:49 AM UTC-4, Rich Jordan wrote: 
> > On Friday, April 8, 2022 at 6:24:37 AM UTC-5, geze... at rlgsc.com wrote: 
> > > On Thursday, April 7, 2022 at 7:24:03 PM UTC-4, Rich Jordan wrote: 
> > > > Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP. 
> > > > 
> > > > Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability... 
> > > > 
> > > > Thanks for any info 
> > > Rich, 
> > > 
> > > As Robert noted, DECnet Phase V over IP is a viable choice. In situations where the client did not want to do that change, I have used sftp quite effectively, often in concert with ZIP/UNZIP. 
> > > 
> > > As an example, in one quick test, I was able to transfer more than a terabyte of data in a reasonable time (I do not have the precise numbers in easy reach at the moment). The context was non-tape backup. The process was: 
> > > - Do BACKUP/IMAGE to a scratch device 
> > > - use ZIP "-V" to compress the BACKUP save set. Your mileage will vary, I was able to get approximately 90% size reduction. 
> > > - Use sftp to copy the resulting ZIP files to the destination machine 
> > > - UNZIP the transferred files 
> > > - use BACKUP to restore the saveset on the destination machine 
> > > 
> > > The above was run in multiple BATCH jobs on both sides. 
> > > 
> > > - Bob Gezelter, http://www.rlgsc.com 
> > Thank you all for responses. I'm not excited about the thought of installing Phase V mainly because it has been 25+ years since I even looked at it. I can see if it is an option. At least that I can test here at our office. 
> > 
> > Losing current DECnet transit is not negotiable apparently; we lost that argument but that choice is with the customer and their network vendor/support; we and the VMS boxes are a side issue that just needs to deal with the change. 
> > 
> > As noted we already do some data moves by doing backup to local disk savesets (using the /DATA_FORMAT=COMPRESS option) then FTP'ing those sets to a PC. Fully tested so we know we can retrieve those savesets, fix the attributes, and restore them because doing even a zero compression encapsulating ZIP takes too much time for the larger savesets (ZIP would preserve the backup saveset file attributes). 
> > 
> > The current VMS to VMS is done with DECnet copies of individual files, or backup of folders/trees or bulk files to remote saveset via DECnet that gets restored on the remote system.
> Rich, 
> 
> If you are transferring trees or subtrees, the /IMAGE is completely unnecessary. 
> 
> The reason I used ZIP was to preserve the OpenVMS file attributes when transferring. OpenVMS ZIP/UNZIP can preserve file attributes without problem. 
> 
> How large are the data volumes being transferred.
> - Bob Gezelter, http://www.rlgsc.com

Sorry for delays, got pulled into someone else's mega-project for a while.

To be clear the image backups are made and transferred to a PC server so they go offsite.  Data transfer from one VMS machine to the other is done using non-image backups, then transfer the saveset, or if there is only a small batch of files, backup with the target being the remote system using proper syntax, or even just copy.

Its not impossible that we might extremely occasionally transfer an image backup to the remote system (or make the image backup with the saveset on the remote system) but that would be a serious recovery operation, not daily data updates.

We only used ZIP on backup savesets to preserve saveset attributes when they were transferred to PC servers, not for any VMS to VMS usage.

 I know ZIP can be used to create a local archive of files or folders/trees with all file attributes saved (then we transfer the archive); I guess its worth trying but there is the _occasional_ need to update one or two major files using /ignore=interlock to get them to the backup system.  In these cases the file is held open by processing until their staff fixes the problem that caused the hold but no changes are occurring so the data is consistent.  THis can happen when processing runs into a problem, and the customer's staff is alerted and probably working the issue but we still need to get that file moved.  ZIP can't do that.

And no, telling their devs to change that isn't going to go anywhere.  Gotta work with what we got.

Still wishing they'd jumped for a cluster license and shadowing.