[Info-vax] Kernel Transplantation (was: Re: New CEO of VMS Software)

[Stephen Hoffman]

On 2024-01-05 04:40:52 +0000, Lawrence D'Oliveiro said:

> On Thu, 4 Jan 2024 22:09:12 -0500, Arne Vajhøj wrote:
> 
>> I was not comparing "Linux port to Alpha" with "VSI actual port of VMS
>> to x86-64" but to "hypothetical port of VMS to being on top of Linux
>> kernel".
> 
> Remember, I’m not talking about porting the whole of VMS, just the part
> that users care about: userland executables and DCL command procedures.
> That’s it.


For some idea of relative scale for that "that's it", that's "merely" a 
sizable chunk of what is roughly 35 million lines of code written in a 
mix of assembler, BLISS, and C with proprietary extensions and API 
calls, and which is entirely dependent on the rest of the 35 million 
lines of code and some unique compilers. Not a small project.



There have been various discussions about this kernel port in a time 
and place that no longer exists too, but that's all fodder for another 
time and another place, and maybe with a little more included below.



Ponder how much of the "upper level" APIs and tools here tie into the 
XQP and ACPs and device drivers, including the terminal drivers, 
network drivers, and storage drivers. There have been ongoing I/O 
issues (e.g. SSIO, quorum I/O) underneath clustering and assumptions of 
storage writes too, and I'd expect those issues to appear elsewhere 
when porting to a different kernel.

It's an immense project. FreeVMS made an effort in this direction some 
years ago, but effectively ran out of staff (volunteers) and funding. 
Lost their domain, too. DEC did a partial port to Mach years ago as an 
advanced development project, but that work was far from complete.

Sector 7 has been porting APIs incrementally for decades now, and 
Sector 7 can have the option of reworking the app source code as and 
where needed.

Valve has a tougher problem, as they can't rework the app source code 
to run on Steam Deck. Valve and the open source projects involved have 
in aggregate done an immense pile of work to get a number of games 
working, though there are many that don't work, and pretty much any 
games with anti-cheat won't.  https://www.steamdeck.com/en/verified  
Apple has expended some development effort in this area too, with their 
game-porting tools: https://developer.apple.com/games/

Could VSI do something akin to FreeVMS, Sector 7, Steam Deck, or Apple, 
and their respective tooling, but starting with the original OpenVMS 
source code? Sure. Then existing OpenVMS customers then have another 
five or ten years of wait to enjoy and with no particular enhancements. 
And then quite probably a whole pile of app-level workarounds for 
whatever didn't get ported or implemented, or that had to diverge for 
reasons.  Or waiting for a yet larger and longer and more complex 
project to allow binaries to run directly, work which would necessarily 
lag behind the rest of the porting work.





What does this kernel swap mean? VSI spends years creating an 
inevitably-somewhat-incomplete third-party Linux porting kit for 
customer OpenVMS apps, and the end goal of the intended customers then 
inexorably shifts toward the removal of that porting kit, and probably 
in the best case the whole effort inevitably degrades into apps ported 
top and running on VSI Linux. Or to porting to and running on some 
other not-VSI Linux. That's certainly a service business opportunity, 
providing customers assistance porting their OpenVMS apps to VSI Linux. 
It does get VSI out of maintaining a kernel, but does not reduce much 
else. And that at no small cost and no small investment, and at a cost 
of a number of other opportunities.




TL;DR: The kernel isn't a big hunk of the ongoing development effort, 
once the port is complete. Yeah, it takes a while to get to a working 
bootstrap and working kernel during a port, though operating as a guest 
reduces that somewhat. Porting to a different platform supported by the 
shared kernel would get easier, though VSI still has to drag along 
compilers and other tooling, or work to expunge that.  De-kerneling the 
userland would be a larger effort. And re-kerneling means VSI must now 
track changes to their chosen replacement kernel, because y'all just 
know some kernel changes will almost certainly be required here.  In 
the best case with re-kerneling, the customers then get a decade of 
delays, with few enhancements, and all for an OS and APIs that arguably 
haven't seen appreciable enhancements and new features since before 
Y2K. Or customers can choose the existing OpenVMS x86-64 guests, and 
can get back to whatever they were doing, and VSI can get back to 
working on enhancements and updates and performance.





>From another time and place, a DEC Usenix paper from way back in 1992 
discussing a kernel-swap project: 
http://fossies.org/linux/freevms/doc/Usenix_VMS-on-Mach.PS

>From a reference to that work: "In 1992, a development team from 
Digital Equipment described a proof-of-concept implementation of VMS on 
Mach 3.0. ... Their work provided independent confirmation that 
multiple OS personalities could be supported on the Mach microkernel. 
At the same time, it exposed certain limitations. For example, it 
proved impossible to accurately emulate VMS scheduling policies using 
Mach. As another example, it was not possible to emulate VMS’ strong 
isolation of kernel resource usage by different users. Preliminary 
measurements also suggested that layering VMS on Mach resulted in 
unacceptable performance overhead. Due to these technical concerns and 
other nontechnical considerations, Digital Equipment did not follow 
through with a production quality implementation of VMS on Mach."

That prototype work predated L4Ka and such, which reduced the overhead 
involved with Mach.





Would I like to see an OpenVMS port to Linux, L4, or otherwise? Sure. 
Fun project. But who wants to buy that? And for how much?






-- 
Pure Personal Opinion | HoffmanLabs LLC