[Info-vax] CRTL and RMS vs SSIO

[Stephen Hoffman]

On 2021-10-09 10:19:42 +0000, Greg Tinkler said:

> On Saturday, 9 October 2021 at 6:00:48 pm UTC+11, Vitaly Pustovetov wrote:
>>> So the main issue is file IO, so change CRTL to use RMS.> >> > gt
>> CRTL uses RMS for file I/O. But there is an issue with concurrent 
>> access of multiple processes to the same file in stream mode. And we 
>> had a choice - 1) rewrite half of Postgres by inserting file locking; 
>> 2) add a new SSIO (Shared Stream IO) service to VMS.
> 
> Sort of.
> RMS worked and has been working of 40+years and does not have these 
> concurrent access issues!  NB there is no such thing at an OS level of 
> stream anything, every thing is clumps of data being buffered is some 
> way, the API that accesses that data from the higher levels may be 
> stream based.  In this case it is CRTL's role to translate the clumps 
> of data into/from stream API.

RMS is a pretty good database, for its time.  Alas, its become rather 
more dated, with an API design that is complex and limiting, and in 
competitive terms RMS is badly feature-limited.

If you need a key-value store and where the developer entirely owns the 
fields used within the punched cards, and where y'all can fit your 
files in 2 TiB (or bound volume sets, gag), RMS is still a fine choice.

For stream access to data, removing the punched-card assumptions and 
file and cluster locking and the rest of effectively removes all of RMS 
from the discussion; in such a case, RMS really isn't used either in 
name, or in general.

As for whether or not there are streams of data, the abstraction 
exists. The difference at the app level is whether the operating system 
and its default file system enforces the use of a punched-card 
abstraction. C does not expect that. Classic OpenVMS apps do.

> So the other choice, 3), fix CRTL to use RMS correctly, and the 
> problems will go away.  Engineering effort would not be great. I don't 
> have access to the code base, but assuming that stdio uses unixio then 
> it is fixing 5 routines.  This would also allow all the other ports to 
> work with minimal changes in the file access area.  If you what to know 
> more contact me directly.

Punched cards and punched-card-based assumptions are rather more 
pernicious within OpenVMS and clustering, and mailboxes, and various 
other areas, alas. For those of us steeped in OpenVMS, the effects of 
these assumptions can be invisible.

> Longer term the SSIO may be useful to RMS, which is where it belongs.

Longer-term, SSIO belongs in XQP, and RMS needs a demotion to "just 
another of the available databases on OpenVMS" atop the XQP, and/or 
atop some replacement XQP and/or FUSE for different file systems, and 
this with various other common databases present.

SSIO and similar work aside, that demotion of RMS and the related and 
substantial investment in new file system and database work are not 
going to happen any time soon. Getting the device drivers and XQP to 
64-bit storage addressing was reportedly one part of the work involved 
(and was once targeted for V8.5), while getting RMS to 64-bit 
addressing was a separate and subsequent feature. Getting RMS to 64-bit 
storage addressing was and is and will be a rather larger investment, 
too. Both VAFS and GFS have been discussed here, but VSI has been busy 
with and increasingly focused on the port and port-related work.

SSIO is unrelated to the other file system work pending here.

Back to RMS and SSIO, apps that don't expect punched-card semantics can 
and variously do perform their own coordination, so sharing the 
underlying files with apps that do expect punched cards is unnecessary, 
and counterproductive.

> Sorry if the above is a little blunt, I appreciate the efforts people 
> have put in over the years, but some of us have using and coding VMS 
> for a very long time, and I really want VMS to be successful and easy 
> to port to.  This has been a good opportunity for me to look more into 
> CRTL and RMS, and see the problems that have been there for decades.

Part of the problem was that thirty years ago, senior DEC leadership 
and OpenVMS development leadership was unwilling or unable to foresee 
the directions that computing was headed, and fallout from that era 
continues to reverberate through to this day around C and IP and RMS. 
And around where OpenVMS has found itself in recent years. As long as 
we're being blunt.

One of the problems that OpenVMS has here is RMS. While RMS was and is 
very useful, it's just not a competitive database in 2021, and too many 
of its punched-card assumptions have permeated the platform. That, and 
the primary RMS API is just bad for making any sort of significant 
changes. This is another area very much like the addition of 64-bit 
virtual addressing on OpenVMS; where providing compatibility for 32-bit 
virtual apps makes an already complex environment (RMS) vastly more 
complex (mixed 32-bit and 64-bit storage addressing within RMS).

> locking is an interesting area, I still feel the current DLM is more 
> than capable of doing the 'lock a byte range' in a way that can be used 
> with the current RMS locking.  Longer term DLM needs some changes but 
> they are about sizes of resource names, scoping of resource names, 
> ability to scan for children resources by name.

C and DLM already implement range locking on OpenVMS.




-- 
Pure Personal Opinion | HoffmanLabs LLC