[Info-vax] Streaming a File on OpenVMS with Caché

[Stephen Hoffman]

On 2015-01-14 23:19:59 +0000, Mack Altman III said:

> You are right I was short of details.
> 
> OpenVMS v8.4
> Cache 2010
> 
> That said, the issue is dealing with Cache 2007 having changed theirs 
> %Stream requirements for OpenVMS to expect the file being read into the 
> stream as a STREAM type, where it was not required prior.

Might want to have a chat with the Intersystems folks directly, then?  
There's not very much Caché-related discussion traffic here in the 
comp.os.vms newsgroup.

> In regards to file types, record formatting and the current age we are in,

Many of the expectations and constructs within VMS date from the 1970s, 
and various key parts of VMS haven't changed all that much.  Which 
"current age" are we discussing?

> is it regular in the OpenVMS world to go through the "guessing game" 
> (if you will) in trying to understand the structure of a file.

VMS has metadata to describe the file format and record format.   Most 
systems have something similar, either generic within the operating 
system or within the applicable database package or application 
involved.   At times, OS X was dependent on metadata located on a 
resource fork, for instance.   When you get a database or Caché 
involved, that environment also has its own ideas and expectations.   
When you have other operating systems and tools involved, those too 
have their own ideas and expectations.

File transfers inherently involve mapping the details of the file and 
record formats from each system across the systems and environments 
involved, and which can be an inexact process; there are usually some 
corner cases.   Of all of the common tools that work with files, SQLite 
does well at allowing the same database file to be transferred around 
and then successfully accessed by any of the supported platforms, and 
zip (zip "-V" on VMS — and you either need extended parsing enabled, or 
you need to quote that "-V" switch on VMS) does well protecting the 
file format metadata during heterogeneous network file transfers.

Windows, Linux, OS X, VMS and other systems all get into trouble 
mapping differences in record formats and file metadata, too.  This 
involves rather more than VMS.  There's not necessarily an exact match, 
nor a fully-reversible, lossless attribute mapping.  Windows and Unix, 
for instance, will strip off the metadata that makes a VMS indexed file 
work.  Backup savesets, well, see the 
RESET_BACKUP_SAVESET_ATTRIBUTES.COM tool.  Windows, of course, sees a 
.COM file type as a binary executable, and VMS as a sequential DCL 
procedure.  All sorts of "fun" lurks here.

Related:
<http://labs.hoffmanlabs.com/node/302>
<http://labs.hoffmanlabs.com/node/357>
Beware: ongoing network outages.

> I ask as this goes on every time we have a new client give us a file, 
> and personally, it's exhausting.

Switch tactics, then?   Create and integrate tools into the customer's 
environment that allow the customers to know little or nothing, but to 
properly package a file for transfer, and to perform a secure, 
encrypted file transfer to your server.

> At the very least, it seems illogical.

Unfortunately, the mess here can be very logical, albeit arcane.

> Does VMS have the capability to have an FTP user transmit a file, while 
> on the backend restrict any file created by said user to a certain 
> format? If so, could this be achieved without communicating the change 
> to the FTP user?

Your code can do that — within the margin of error involved around 
determining the file format and file validity, of course — but sftp and 
ftp are generic tools.  There's no generic mechanism to block certain 
file formats or filenames with these or most other generic tools.   Put 
another way, consider implementing the tools to package and properly 
secure and perform the data transfer via sftp or encrypted mail message 
attachment or whatever, and provide these tools to the folks you're 
working with.  If you're working with application crashes for instance, 
work within the existing tools to log and automate the uploads.

If I've guessed wrong in the above, please consider describing the 
general problem that you're trying to solve, and some background 
involved.   Please don't start with the solutions and tools you're 
working with here, as that's secondary to my being able to understand 
whatever you're trying to solve here.  It's clear that you're trying to 
transfer a file, and that you're having issues with Caché, and 
apparently somewhat unclear about file formats on VMS, but beyond that, 
well, I'm just guessing...  I don't what you're up to with the stream 
transfers within Caché, nor whether binary-mode transfers might be more 
appropriate.


-- 
Pure Personal Opinion | HoffmanLabs LLC