[Info-vax] The best VMS features, was: Re: openvms renaming file

[Stephen Hoffman]

On 2018-05-29 02:20:53 +0000, Dave Froble said:

> On 5/28/2018 4:08 PM, Johnny Billquist wrote:
>> On 2018-05-28 20:56, Simon Clubley wrote:
>>> On 2018-05-28, Dave Froble <davef at tsoft-inc.com> wrote:
>>>> 
>>>> I don't think so.  RMS indexed files allow concurrent use, using the DLM.
>>>> 
>>>> The problem with RMS isn't the design, it's the implementation.  Modern 
>>>> RDBMS systems do much of their work in memory.  RMS does the work on 
>>>> disk.  Quite a large difference in speed, as one might expect.
>>>> 
>>>> Re-implement RMS in the same manner and you'd see totally different 
>>>> performance.  Ya got to remember how old the design and implementation 
>>>> of RMS is.  40 years ago it worked well.

There've been massive shifts in the hardware.  SSDs replaced HDDs (and 
there are differences there around how blocks are allocated and 
overwritten), and byte-addressable storage is arriving.  And 
expectations have moved on.

>>> RMS ISAM files would still be very limited compared to a database, even MySQL.
>>> 
>>> For example, compare what is involved in adding a new field to a 
>>> database compared to what is involved in adding that same field to a 
>>> RMS ISAM file.
>> 
>> Technically, this is not really a problem in RMS, but one of most 
>> applications using RMS. Most applications are pretty hardcoded how they 
>> expect the file to appear, and so any change will cause big issues.
> 
> Which in my biased opinion is a poor design feature of RMS.
> 
>> The designers of RMS, however, obviously, intended that clever 
>> application writers would not be limited in such a way.

Obviously.  Because a clever developer can get the same 
ease-of-doing-it-all-yourself and extensibility "benefits" from 
performing $qio or $io_perform to the disk, too.

>> Adding a field, as such, is just a question of copying the file,
> 
> There is the problem.  Taking the file offline, and copying it.  As 
> with the database, what values go into the new field?

Ayup.  RMS (also) lacks online backup coordination APIs, mechanisms for 
format conversions, for online tuning, and for reclaiming and otherwise 
cleaning up the remnants of deleted record dreck, and otherwise.

>> and adding that field in all records. There are tools that easily can 
>> do this. RSX comes with such tools in the standard RMS toolbox. I would 
>> assume VMS have similar, or more capable tools.

What tools OpenVMS has are not particularly capable.  All of the cases 
I'm familiar with have needed to write custom tools and/or custom APIs.

> I'm not sure what a database does when one adds a column to a row, or 
> re-sizes a field. I'm actually a bit curious, just a bit. In the 
> database products, it's all very transparent.  No taking the database 
> offline, no copying, and such.  I'm betting there is lots of code 
> behind such capabilities.  LOTS!

There are times when rolling your own conversion and rolling-upgrade 
support is important, when you can get away with not dealing with that 
case and/or can accept an offline conversion.  Or other available 
approaches.  Or when the addition of a dependency on and the 
abstractions of a database are worth the costs or performance or 
whatever other trade-offs might be considered for the app.

>> Applications could be clever enough to actually refer to fields based 
>> on names, instead of just using fixed, known, chunks of the record.

And of course any clever developer can write all that bespoke code in 
machine code, because spending time replacing compilers and frameworks 
is a viable strategy.

>> RMS do have names for the fields, which you can set and read.

Names can exist for indexed file key fields, yes.  Names don't exist 
for not-a-key data fields within a record.  That detail was generally 
left to the clever developer and/or the use of CDD to provide, or the 
use of a non-RMS database that offers that.  SDL was a key part of 
making those sorts of static declarations less tedious to maintain 
across multiple languages, too.

>> And then you could base your access on this information, in which case 
>> a change of the file presents no problem at all.
>> 
>> However, admittedly, I doubt any application (more or less) actually do 
>> this. But do not blame RMS for this.

There really aren't tools or documentation for mixed-version 
configurations, tools for assisting with data reformatting and data 
conversions are lacking.

You're entirely correct in that applications could be clever enough to 
do this.  Though pragmatically, RMS provides little more than what $qio 
provides for mixed-version and migration tasks.

SQLite, PostgreSQL, MariaDB, MySQL and other tools are massively better 
at adding columns and tables.

With RMS, many developers will end up creating multiple files and 
variously parallel tables for these sorts of tasks.  We get a barrage 
of separately-managed SYSUAF, RIGHTSLIST, NET*PROXY, etc., data files, 
and each configured with one of what becomes an increasing number of 
unrelated logical names, and down the rabbit hole we go.

We get differences within app file handling logic when files with the 
version ;32767 limit, too.  Or ;65535, when or if that arrives.  
Whether that needs to be handled with a roll-over, or is an intentional 
design detail of the app for suppressing new file versions for that 
matter, or just gets ignored.

>> If you ever take time to learn how Google's bigtable works, you'll see 
>> that it is almost identical to how RMS works. Quite funny, actually.

Both are key-value stores, certainly.  Beyond that, there are some very 
significant differences between RMS indexed files and Google BigTable.

http://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf 






-- 
Pure Personal Opinion | HoffmanLabs LLC