[Info-vax] HP wins Oracle Itanium case

[Stephen Hoffman]

On 2012-08-22 02:11:29 +0000, David Froble said:

> Stephen Hoffman wrote:
>> On 2012-08-21 20:37:44 +0000, Bob Koehler said:
>> 
>>> In article <zCQYr.12412$7h.10088 at fx23.am4>, ChrisQ <meru at devnull.com> writes:
>>>> On 08/21/12 17:33, Bob Koehler wrote:
>>>>> In article<b9NYr.4322$3p2.4259 at fx03.am4>, ChrisQ<meru at devnull.com>  writes:
>>>>>> 
>>>>>> As for RMS, a file system is to store bytes and any imposed structure on
>>>>>> the data should be layered on elsewhere. So yes, you will have to write it
>>>>>> yourself :-)...
>>>>> 
>>>>> "a file system is to store bytes"
>>>>> 
>>>>> Nope.  Meaningless hype.
>>>> 
>>>> ymmv, but the ideal file system really doesn't want to know about
>>>> various data
>>>> formats. It's main function is to store and retrieve data.
>>> 
>>> Nope.  Disks are made to store and retrive data, filesystems to
>>> organize it.
> 
> I agree with Bob ..

And (clearly) I do not.  I'd rather not write stuff, and the more I 
work with VMS (unlike other platforms) the more I find I have to write 
stuff.

> 
> I'll also point out that for your data, there is no need to use RMS if 
> you desire not to do so.  QIO works for reading and writing data.  
> Mount /Foreign and roll your own if you don't like ODS.

"Rolling my own" is the antithesis of where I'd rather be, here.   I 
did end up rolling my own RMS-calling library, however.

> On the other side, providing RMS allows various languages to access 
> data files without the requirement of providing the underlaying file 
> system.

That much is obvious.  My gripe is that what's available isn't enough.  
 (And beyond sequential configuration files and such, and SQL 
databases, how many folks are actually poking around inside the data 
storage files of an unrelated application?)

> 
>> And then there are the things I find frustrating about the file system,
>> and I'm here lumping VMS mechanisms such as mount, RMS, and the XQP, 
>> together...
> 
> Oh, good, I finally get to be the devil's advocate, not the defense :-)
> 
>> o RMS files don't defragment themselves,
> 
> Short of some Relational databases, is there any file system anywhere 
> that does this?  And even in something that does, can it be done 
> without copying the data elsewhere?

So why doesn't this hypothetical RMS-self-compressing file system 
exist, then?  Wouldn't this be a feature that would want to make you 
upgrade to the latest and greatest VMS?  It'd mean folks wouldn't have 
to either do a whole lot of RMS coding, or take applications and files 
offline for file-packing maintenance.

> 
>> and indexed files don't pack old records.
> 
> Give ya this one, my DAS database system doesn't either.  I've looked 
> at it more than once, and the hassle doesn't seem worth the effort.  
> However, a programmer can re-use a deleted record.

On other systems, I deal with objects, and the storage system deals 
with this icky file and record stuff for me.  I don't know how the 
stuff is stored, and I don't care.  Well, technically I do know and did 
look, but not having to mess with reading and writing records and 
marshalling and unmarshalling the data is _really nice_.

> 
> < The programmer or the end-user has to deal with that mess.
> 
> It was ever thus ..
> Isn't that why we have programmers ??

Yes, and it's why programmers can find existing tools - like RMS - to 
be limited, gnarly, feature-limited and generally a pain in the rump to 
use.  We're never happy with what's available.  It's why we're not 
satisfied with a bad interface, a weak feature set, or a tool that - 
for instance - doesn't provide on-line maintenance.  Why we're looking 
for different or better solutions.  Why it'd be nice if RMS dealt with 
this dreck for me.

> 
>> o RMS files require an extra license and related coding for 
>> transactional processing and rollbacks, and the transactional API 
>> doesn't document any means of synchronization with the other activities 
>> that might be necessary within an application.
> 
> Some people consider marketing decisions a technical issue.  I don't.  
> But I can consider it a royal PITA.
> 
> Sychronizing any database software with an application has always been 
> a tough nut to crack.

The TP processing embedded in VMS (DECdtm) is pretty good.  It's just 
not documented for this particular task; for adding your own activity.  
As for some of the available techniques 
<http://the-paper-trail.org/blog/consensus-protocols-two-phase-commit/> 
or otherwise, if you don't have access to the TP APIs.

> 
>> o RMS files (and disks) don't scale past two terabytes.
> 
> Did any pre-1990 OS?  Maybe ever any pre-2000 OS?  Let's be fair in 
> case we're doing any comparisons.  Or if we're just listing lack of 
> updating when it should happen, well, isn't that what these discussions 
> are all about?  HP not continuing VMS development?

The retrocomputing folks are often interested in what happened in 1990.

The folks that wish to debate what DEC did or did not do, or should or 
should not have done, as well.

If I'm coding in RMS, I'm more interested in this year, the current 
APIs for whatever VMS version I'm using, and going forward.

> 
>> o RMS programming routines have an exceedingly arcane programming interface.
> 
> If calling RMS directly, I got to agree.  If using RMS from a HLL, then 
> I don't agree.  RMS benefits the VMS HLLs.

The API also makes the operations slower, because the processing code 
has to unpack and interpret all of those structures.

> 
>> o RMS sequential files force the user to deal with slightly different 
>> formats, particularly when transferring files around.
> 
> That's more like, the rest of the world came up with something 
> different, and if there was to be interchange of data, RMS needed to 
> implement that used by others.  At least it can adapt.  How well do 
> other OS handle RMS files?  They are the ones having trouble adapting.

If VMS were the predominiate operating system or were in ascendence, 
the Microsoft model of compatibility from the late 1990s ("Our way or 
the highway") work.  But...

> 
>> o RMS relative files lack sparse storage capabilities; you're pushed 
>> over into index files if you need that.
> 
> Is there any method to do this?  If the record can be found only by 
> it's position in the file, then the calculations to find a record need 
> some fixed information, such as record length.  Now, if you're going to 
> have some pointer to each record so it can be found, well, isn't that 
> an index?

Hashed or b-tree files.  Could an indexed file do this?  Sure.

> Are you suggesting that there are data storage models that can have 
> variable length relative records without some type of pointer?  If so, 
> I'd be interested in knowing about them.  That would be very 
> educational.

There are always pointers in a file system.  Relational databases have 
done wads of data for years.

> 
>> o RMS record and I/O data transfers are limited in size.
> 
> As are most things in this universe.  At some time in the past 32767 
> bytes was considered quite large.  Time changes all things.  Did Unix 
> in 1970 have any I'O limits?  If so, and they were modified, then that 
> just takes us back to lack of ongoing development of VMS.

We're in 2012, and tossing around large buffers is much less of an 
issue, and much more efficient than reading a gazillion little records. 
 (Reading the disk file into memory for processing is vastly more 
efficient and massively faster than rummaging around with records.  
This does mean you're subject to power hits in the write path, just as 
happens with VMS and section files.)

> 
>> o RMS files require you to deal with records, rather than dealing with 
>> your data; there are no marshalling and unmarshalling APIs.
>> o Heck, there's no concept of marshalling and unmarshalling anywere; I 
>> have to roll my own data structures with SDL or {whatever}.
> 
> As I'm not sure what "marshalling" is, I'm assuming it's defining and 
> accessing the pieces of a data record.  That's always been one of my 
> complaints about RMS, that the data file system doesn't know a dang 
> thing about what's in the records.

<http://en.wikipedia.org/wiki/Marshalling_(computer_science)>

> 
> The DAS database system I mentioned above contains this "marshalling" 
> information within the file, and the database routines assign labels, 
> if desired, to the data fields in a data record when the record is 
> read.  This dynamic labeling is more labor intensive, but, allows for 
> many utility programs for manipulating the data files, records, and 
> data fields.
> 
>> o VMS (and RMS) doesn't implement application sandboxing.
> 
> Without having huge amounts of memory available, what does?

<http://en.wikipedia.org/wiki/Sandbox_(computer_security)>

> My preference is development and testing systems.

Mine is to harden production, too.

> 
>> o RMS doesn't implement action routines; there are no built-in triggers 
>> for activity, short of repurposing the security APIs.
> 
> Examples?  What would such be used for?

The classic example is triggering some site-local processing for a file 
newly created in a directory.  There are many others.

> 
>> o RMS doesn't have a way to set a default sequential file format.
> 
> Well, it did once, until it had to be modified to support multiple 
> formats.  But yeah, being able to have a consistent default would be a 
> good thing.
> 
>> o RMS lacks an API and storage for application metadata, beyond 
>> repurposing an ACE.
> 
> Ya know, I don't get out much, so, just what would this be, and what 
> could it be used for?

Storing the URL source of a downloaded file, for instance, or the 
version of the tool that created the file.  (Can this be done _in_ a 
file?  Sure.  But not in a fashion that SEARCH (the glacial nature of 
same being a whole 'nother problem) could dig out as part of its 
selection criteria, or that DIRECTORY could show.

> 
>> o RMS lacks a data replication API; HBVS and journaling are it.
> 
> Agreed.
> 
>> o VMS had (has?) a separate shelving package and tools for nearline 
>> archiving and restoring files.
> 
> 
>> o VMS (and RMS) lacks the application synchronization primitives for a 
>> consistent on-line BACKUP.
> 
> I've heard this many times, and I agree that it is a problem.  But how 
> large of a problem.  Assume that you could journal all activity that 
> didn't make a backup, and then included that data with the backup.  
> Sure, you could restore as of the point in time when the backup 
> finished, but for a 24 x 365 application, just how much does that help?

It's one of the ways I've recovered databases after the very expensive 
storage controller fried its entire RAID volume.

> As mentioned before, I don't get out much, so I'm asking, is there any 
> system anywhere that can perform an online backup and always get it 
> 100% right, and if it does, what about subsequent activity?

Ignoring most (all?) relational databases, which support exactly this 
on-line backup with their database-specific tools...

Would this be 100%?  Possibly.  Possibly not.  Depends on the application.

And there would always be applications which did not utilize this hook.

But would this be better than what is available now with 
/IGNORE=INTERLOCK?  Definitely.

> 
>> o RMS lacks a way to modifying the structure of an existing file in 
>> place; you're left to either implement this record translation yourself 
>> at run-time, or rewrite the file.  (This made SYSUAF upgrades all the 
>> more interesting, as there was no good way to do that across versions.)
> 
> And the reason DAS has this offline capability is to allow quick and 
> easy modification.  It's still copying data from a source to a 
> destination.

Relational databases can typically do these changes "hot".  Online.

> 
>> o And VMS doesn't support variant file systems; there's no API for 
>> inserting your own file system underneath RMS, or a replacement for RMS.
> 
> As for file storage, you're correct.  Does any other OS allow multiple 
> file systems on the same disks?

VMS supports BFS (Bridge File System; the TCP/IP Services "stuff"), 
NFS, ODS-2, ODS-5 and (for a while) Spiralog, and BFS can coexist on 
the same disk.  And using the LD logical disk driver, even VMS can 
support different file systems on the same disk.  WIth other operating 
systems, the file systems can and often do default to the use of 
partitioning, which allows the same physical disk or the same virtual 
disk to have several different file systems.  (This partitioning is 
part of the EFI specs, too, and VMS had to do some hackery to hide the 
partitioning and allow its non-partition-capable I/O stack to operate.) 
  Unix commonly supports several different file systems, in different 
partitions on the same disk.

The concept of the disk is getting quaint, too, as all the applications 
care about is a pool of blocks, and who cares what physical storage is 
involved.  (As a recent example, there's Amazon Glacier, which is a wad 
of very cheap, very durable, storage somewhere in the cloud.  
<http://aws.amazon.com/glacier/>)  But I digress.

> I frankly don't see how this could be done.  One file system per disk, 
> yes, I can see that, and that is might be useful, if the tools to use 
> multiple file systems existed.

Beyond partitioning, a FUSE (file system in user space) allows you to 
do this stuff in user space, too, which is handy for developing the new 
file system, for testing, and for isolated use.  (You can go as far as 
running Linux in user space these days, too; running a whole operating 
system as an application.)

> 
> I guess an example would be Weendoze handling both NTFS and FAT.  I'm 
> assuming that there is different code for each.  Or possibly different 
> code for certain operations.
> 
> Would ODS-2 and ODS-5 be considered different file systems?

They are, in fact, different.  ODS-5 is a proper subset of ODS-2.  
They're not very different, but they are different.

> 
>> o Add a key to an existing indexed file.  Or remove it.  Go ahead.  Try 
>> it.  Make my day.
> 
> That was always another of my problems with RMS.  In an indexed file, 
> the bottom level keys were actually the data record.  To change the 
> primary key, you needed to move the entire data record.
> 
>> o RMS forces me to read records in, and to write them out.  Personally, 
>> I'd rather let RMS deal with that, and just hand me the data.  (See 
>> marshalling, et al)
> 
> Like paging, huh?

Like swapping the whole file in.

> 
>> o No built-in tools to assist with file format and data conversions, 
>> beyond CONVERT /FDL and friends.
>> o It's slllllloooooowwwww.
> 
> Anything that resides on disk is slow.  Cache it all up in memory and 
> you'll see a difference.  RMS was designed and implemented in a time 
> where memory was measured in KB, not MB, and GB was unthinkable.

Did I not mention that RMS was pretty good in the 1980s?  And that 
we're now decades past that.  And that a single laptop might have as 
much physical memory and more SSD storage than the cluster of 
AlphaServer systems I recently installed had for physical memory and 
rotating-rust storage?

> 
>> And I am sure I can find a few more things that annoy me about RMS or 
>> the XQP here, too.
>> 
>> Things that don't annoy me about RMS?  RMS is wicked-good at not 
>> corrupting itself.  Which is very important.
>> 
>> RMS was a great idea well into the 1980s.   And it's a marvelously 
>> well-engineered system.  But it's really gotten quite feature-stale in 
>> the ensuing decades.  And it lends itself to the one-size fits all 
>> approach of application design, which usually then leads to busted 
>> applications, difficulty with backups, and other messes.
>> 
>> And yes, there are a number of applications that can (and do) (and 
>> should) still use RMS.  I have a number of them, and will likely have 
>> more.
>> 
>> But in the end, I find the Unix approach of allowing me to layer on 
>> whatever system I need much more flexible.  (The days of rolling my own 
>> record-level storage system are largely gone, except when I'm working 
>> on VMS with RMS, and without the benefit of a database in the target 
>> environment.)
> 
> I can appreciate the concept of having more than one file system 
> available, and more than one database system available.  On VMS I've 
> practiced that, using both RMS and DAS.


VMS doesn't have anything past RMS integrated, and RMS is limited, 
which is where I get into trouble with my preferences.  With other 
platforms, there are integrated databases.  On thinking about it, it's 
that RMS more and more gets in the way these days, and is often (much) 
less capable than what's available on other platforms.

Some future version of VMS, with an up-rated RMS, or with integrated 
SQL and noSQL databases?  Might get some folks to upgrade.  Might just 
get some others to stay on VMS.  Etc.



-- 
Pure Personal Opinion | HoffmanLabs LLC