[Info-vax] The best VMS features, was: Re: openvms renaming file

[Johnny Billquist]

On 2018-05-29 17:46, Stephen Hoffman wrote:
> On 2018-05-29 02:20:53 +0000, Dave Froble said:
> 
>> On 5/28/2018 4:08 PM, Johnny Billquist wrote:
>>> The designers of RMS, however, obviously, intended that clever 
>>> application writers would not be limited in such a way.
> 
> Obviously.  Because a clever developer can get the same 
> ease-of-doing-it-all-yourself and extensibility "benefits" from 
> performing $qio or $io_perform to the disk, too.

True. You can replicate RMS on your own as well, if you want to. It's 
always a question of finding the right compromise between flexibility, 
performance, and time to develop.

>>> Adding a field, as such, is just a question of copying the file,
>>
>> There is the problem.  Taking the file offline, and copying it.  As 
>> with the database, what values go into the new field?
> 
> Ayup.  RMS (also) lacks online backup coordination APIs, mechanisms for 
> format conversions, for online tuning, and for reclaiming and otherwise 
> cleaning up the remnants of deleted record dreck, and otherwise.

This is something I didn't know, and which surprised me. RMS-11 
certainly provides such tools.
Not the backup coordination, but the other parts... Maybe a bit crude 
tools by todays standards, but they certainly exist.

>>> Applications could be clever enough to actually refer to fields based 
>>> on names, instead of just using fixed, known, chunks of the record.
> 
> And of course any clever developer can write all that bespoke code in 
> machine code, because spending time replacing compilers and frameworks 
> is a viable strategy.

Yes, they could. Or they could make use of capabilities that RMS already 
provides.

>>> RMS do have names for the fields, which you can set and read.
> 
> Names can exist for indexed file key fields, yes.  Names don't exist for 
> not-a-key data fields within a record.  That detail was generally left 
> to the clever developer and/or the use of CDD to provide, or the use of 
> a non-RMS database that offers that.  SDL was a key part of making those 
> sorts of static declarations less tedious to maintain across multiple 
> languages, too.

Ah. Good point. Yes, only key fields have names. Not arbitrary fields.

>>> And then you could base your access on this information, in which 
>>> case a change of the file presents no problem at all.
>>>
>>> However, admittedly, I doubt any application (more or less) actually 
>>> do this. But do not blame RMS for this.
> 
> There really aren't tools or documentation for mixed-version 
> configurations, tools for assisting with data reformatting and data 
> conversions are lacking.
> 
> You're entirely correct in that applications could be clever enough to 
> do this.  Though pragmatically, RMS provides little more than what $qio 
> provides for mixed-version and migration tasks.

Like I said. Programs in general do this in the most simplistic way, 
which is probably the biggest problem to any changes to the format of 
the files.

> SQLite, PostgreSQL, MariaDB, MySQL and other tools are massively better 
> at adding columns and tables.

Agreed. Modern tools have much better abilities to do this.

> With RMS, many developers will end up creating multiple files and 
> variously parallel tables for these sorts of tasks.  We get a barrage of 
> separately-managed SYSUAF, RIGHTSLIST, NET*PROXY, etc., data files, and 
> each configured with one of what becomes an increasing number of 
> unrelated logical names, and down the rabbit hole we go.
> 
> We get differences within app file handling logic when files with the 
> version ;32767 limit, too.  Or ;65535, when or if that arrives. Whether 
> that needs to be handled with a roll-over, or is an intentional design 
> detail of the app for suppressing new file versions for that matter, or 
> just gets ignored.

Yup.
But I still don't think it is correct to just blame RMS for this. RMS 
gave a fundament upon which something good could have been done. But 
this was never realized.

>>> If you ever take time to learn how Google's bigtable works, you'll 
>>> see that it is almost identical to how RMS works. Quite funny, actually.
> 
> Both are key-value stores, certainly.  Beyond that, there are some very 
> significant differences between RMS indexed files and Google BigTable.
> 
> http://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf 

Oh. I know pretty well how bigtable works. The big difference would be 
the third dimension of the timestamp. Apart from that, we're just 
talking about the scale of things. You can have way more rows in 
bigtable than you'd ever want to throw at RMS, and way larger records, 
and each field can be way larger. But that is just a difference in 
scale. Not concept.
The automatic partitioning and distribution is not really anything that 
is relevant to the concept. That is just how it scales up. Don't make 
any difference to how you actually use it.

Apart from that, you have rows, which you refer to by name. Each field 
is just a blob of bytes, and it's just sorted in the most obvious way.
And you can read through it on each field, and get the rows in order 
based on that field. Just as with indexed keys in RMS. If every field is 
a key in RMS, you'll have the exact same thing as bigtable, if we ignore 
the timestamp dimension, and the scale.

   Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol