[Info-vax] Open Source on OpenVMS (was: Re: Taking a break...)

[Stephen Hoffman]

On 2022-06-16 13:50:16 +0000, seasoned_geek said:

> On Thursday, June 16, 2022 at 7:15:37 AM UTC-5, Simon Clubley wrote:
>> I am surprised that no-one else has offered any comments about this or 
>> about how they see, at a technical level, the best way forward for 
>> porting open source software to VMS.

There's a whole lot known about what's required. Fundamentally, it's 
people getting paid to work on open source for OpenVMS, and investments 
from VSI in fixes and updates. In short, money.

> Well, as someone who had to write something to catch all OPCOM and some 
> other messages creating modern syslog format (forget the number then) 
> using C when the C compiler was about 10 years out of date so had to 
> port the bulk of the GNU string library into the project before I could 
> get anything to work . . .

Good news: while the OpenVMS C compiler reached C99 support back around 
2020 and is thus still a decade or so out of date, part of that will be 
changing with the OpenVMS x86-64 port, LLVM, and Clang.

As for syslog, there's an old port at VSI: 
https://vmssoftware.com/products/syslogd/

And a whole lot of GNU and Clang and POSIX extensions are missing, 
though base C support is vastly better than it once was given C99. 
getopt_long is probably the one API I've bumped into most, and having 
to avoid the missing calls is less than fun for new work.

> There isn't one.

There's way more here to the dependency chains involved, yes. That was 
all starting to pile up around Y2K, and the dependencies have only 
increased. The scale and scope of app development has increased, along 
with developer and customer expectations.

> Operating systems and written on the 
> x86-wanna-be-a-real-computer-one-day-when-I-grow-up platform never 
> scale up.

x86-64 servers are substantially more capable than the Alpha and 
Itanium boxes that many of us are dealing with.

How fast OpenVMS and its memory management model will perform on x86-64 
remains to be determined, though.

> You can scale a real OS and real software down because you are taking 
> things out, but the x86 platform is inherently so insecure and 
> non-robust you can't really port anything of significance to OpenVMS.

Alas, scaling down is surprisingly difficult, particularly around 
memory and power requirements.

Might want to encourage housekeeping interrupts to all hit at the same 
time to improve power efficiency.

There's a whole lot of OpenVMS that just doesn't make sense in 
low-range configurations, too.

But if you're thinking of x86-64 servers as small or slow or limited, 
that's not the case.

But as for OpenVMS itself, that never got around to adding support for 
the larger HP/HPE Itanium servers in years past, and is decidedly now 
positioned in the low-to-mid-range server computing market in recent 
years.

> Despite what many others in here may wish to promote, layered logical 
> name tables with user, group, and ACL access security is a robust and 
> amazing security feature when properly used. Applications using these 
> features in an appropriate manner can be self-protecting with the 
> default super user tree pointing off to an "entertainment value" area 
> of the system that can send a security alert and keep the user poking 
> around for hours while teams/authorities backtrace the connection. You 
> have to "know the magic codes" to get out of the "entertainment value" 
> tree into the actual tree where all of the actual accounting, customer, 
> cc, etc. information is.

You've also just described LDAP.

Pragmatically, all of the logical name stuff could be re-hosted over 
onto LDAP and with minimal or no disruption, and with support for 
multiple hosts and replication added.

> Many didn't take it to that level, but you could.

Most will use parallel LDAP, if they want or need full separation.

> The best, won't say where, won't say when, were the sites that 
> re-purposed the "Test Cluster" as a glorified terminal.

These same messes can arise with OpenVMS, and this mess is unrelated to 
the underlying processor architecture.

> All of the real software was loaded on this system. It was used for 
> integration testing and new user training. It even had regularly 
> scheduled nightly jobs. The operators sneaker netted actual inventory 
> files over to the system each night. On the "Test Cluster" the critical 
> files all had generic names like INVENTORY.DAT, INVOICES.DAT, etc. 
> Completely different names on the production cluster.

Outside of its own non-production data, there's no reason for the test 
environment to diverge from productions, and often various good reasons 
why it should not. The more test diverges, the more time spent testing 
divergences.

> The production cluster had no outside access. To get a terminal on it 
> you had to go through the "Test Cluster" with your magic-magic access 
> code to a magic-magic program that would open a basic terminal 
> connection to a production cluster on a different network. It was a 
> specific non-generic terminal package that would only connect with 
> itself, I don't remember what. They might have rolled their own.

A combination of access controls and DNS are commonly used to direct 
clients to the appropriate servers.

> The only "outside access" the production cluster had was via MQSeries 
> queues from a Websphere server. There was no way to initiate any kind 
> of file transfer or terminal connection from the outside world. All of 
> the free-flowing XML/JSON/etc. messages were converted to fix-width 
> fixed-field-width proprietary message formats before being placed on 
> the queue.

That's server isolation is a fairly common preference, and independent 
of the processor architecture.

> I took that tiny detour to point out the level of thinking that want 
> into many/much/a-not-insignificant-part of the VMS world. People 
> actually thought about things like bifurcated terminal applications and 
> other secured communications. We have $SEVERITY with many different 
> values that qualify as $SUCCESS because the values were deemed 
> INFORMATIONAL.

None of what you've described so far is unique to OpenVMS. As for the 
OpenVMS condition handling, yes, that can be useful. Though more often 
than not lately, I'm selecting by error, and not on the OpenVMS SIWEF, 
and logging and exiting on unrecognized errors. Unix shell error 
handling is different, subtle, and quick to anger. Other environments 
and other platforms differ.
> 
> The x86 world doesn't have this. More importantly can't be made to deal 
> with it. To make OpenSource software of any significance run correctly 
> you end up with one-off forks because what you need can't be passed 
> back upstream.
> Yes, many small useful things like JED https://www.jedsoft.org/jed/ can 
> be ported. I use JED a lot in my embedded systems work.
> 
> Instead of having huge range of exit codes that can convey $SEVERITY 
> one can only safely use 0-125 and 0 **is the only success value**

Oh, you really need to keep up with that particular mess. 0 can 
sometimes be the only success value, or negative can be failure and 
positive success, or other schemes. That's when the APIs and tools are 
still using the old OpenVMS-style APIs.

Some other error reporting via error classes are far more flexible, for 
newer environments.

Here's the older NSError, as an alternative to what OpenVMS does: 
https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/ErrorHandlingCocoa/ErrorHandling/ErrorHandling.html 


As for OpenVMS, one of my favorite" error messes is the stupidity 
around errors and parameters for system errors, for RMS errors, and for 
user errors. Why that isn't consistent is a discussion for another era, 
and changes there are ~impossible due to compatibility. But I usually 
end up wrapping the built-in error handling and message output support 
to make it somewhat less stupid.

And again, this design (or this mess) has nothing to do with the 
underlying processor architecture.

> https://unix.stackexchange.com/questions/418784/what-is-the-min-and-max-values-of-exit-codes-in-linux 
> 

Being quite portable, Linux specifically avoids tying into x86-64 
architecture, outside of that necessary to boot and build for that 
architecture. Put differently, what Linux does for error codes is what 
Linux does, independent of x86-64 architecture.

> The one package that shoulda-woulda-coulda made a lot of sense was 
> PostgreSQL because it actually started out on VMS decades ago. 
> According to the port page
> https://wiki.postgresql.org/wiki/OpenVMS_port
> last updated 2012 . . .

PostgreSQL hasn't ever supported OpenVMS. (You're probably confusing 
PostgreSQL with Ingres or some other database package which did have 
support for VAX/VMS or OpenVMS.)

The PostgreSQL port stalled due to shared stream I/O corruptions (SSIO) 
latent in OpenVMS, among other issues.
VSI is reportedly addressing those corruption issues for single-host, 
non-clustered apps.

SSIO info: http://de.openvms.org/TUD2012/opensource_and_unix_portability.pdf

And for open source in general, there are both parts of OpenVMS as it 
exists now that are open-source, add-ons which are open-source, and 
there are more parts on the way and not the least of those is LLVM.

> I say it would have made a lot of sense only because OpenVMS could then 
> become a robust host OS for a robust relational database that all of 
> the x86 languages that matter could hit. Would have granted much 
> purpose to the x86 port, especially if the COBOL/BASIC/FORTRAN/C 
> compilers got a PostgreSQLMod utility so shops could begin migrating 
> applications from RDB to a free-as-in-beer database the new maintainers 
> of VMS could sell support contracts for.

VSI does seem intent on a port of PostgreSQL. Probably as an option or 
fallback, should Oracle Rdb be delayed or unavailable or otherwise 
under- or uninteresting to VSI customers.

The integration of that and of SQLite would provide substantial 
upgrades to OpenVMS, too. zip and unzip too, but I digress.

> I suspect that was a giant hill to push a boulder up though.

Definitely. OpenVMS app ports from Rdb to not-Rdb will involve a fair 
amount of work, and the pre-compilers are just part of that "fun". 
(I'll have to ponder what's involved in pre-compiler support, given 
LLVM-based languages. LLVM integrates far better than did DEC-era 
compilers.)

> Lack of any real desktop for VMS torpedoed adding support to a large 
> C++ framework like CopperSpice or Qt that would have hidden most of the 
> ugly OS details

Absent massive and likely futile investments, OpenVMS on the desktop is 
and will remain uninteresting outside of the installed base, and 
outside of those few hobbyists with unusually low app expectations, and 
outside those not requiring interoperation with other common desktop 
apps and formats. Insufficient Phillips for meaningful profits.


-- 
Pure Personal Opinion | HoffmanLabs LLC