[Info-vax] What does VMS get used for, these days?

[Jake Hamby (Solid State Jake)]

On Sunday, November 6, 2022 at 3:42:16 PM UTC-8, Craig A. Berry wrote:
> On 11/6/22 12:58 PM, Arne Vajhøj wrote: 
> > On 11/6/2022 8:55 AM, Scott Dorsey wrote: 
> >> First thing: VMS has heavyweight processes.  There's a lot of stuff in 
> >> the 
> >> process, so spawning off new processes takes a good while, and you 
> >> don't do 
> >> it very often.  Conceptually different than Unix and Unixalikes where the 
> >> processes are lightweight and the overhead of a fork is minimal so you 
> >> fork 
> >> off a new process for nearly everything. 
> > 
> > That was the gospel for many years. 
> > 
> > But the overhead of creating a process much be a lot less 
> > significant on an Itanium or x86-64 today than it was on 
> > a VAX 35 years ago. 
> > 
> > And I believe that for true high performance then even 
> > *nix are switching from traditional forking to threads.
> And not everyone thinks fork() is such a great idea anymore even aside 
> from performance considerations, e.g.: 
> 
> https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf 
> 
> This could be good news for VMS if major applications start using 
> posix_spawn() instead of fork(); it would map much better to the VMS way 
> of doing things, and it might even be reasonable to provide a 
> posix_spawn() API on VMS.

I was going to share that paper if someone else hadn't. fork() on UNIX is much slower than people think. I'm testing performance of several shells as part of a hobby project that grew out of hand porting GNU bash from K&R-style C to C++, and if you download "shellbench" from GitHub and run the subshell test, you'd be surprised how few subshells/sec, or external process controls (fork/exec) per second you get.

Solaris ksh / ksh93 uses posix_spawn() and Solaris 11.4 implements in in the kernel instead of using vfork() like it was doing previously. For those who don't know, VMS has vfork() that works more-or-less like BSD vfork(), but implemented with something like setjmp/longjmp, which passes the open file descriptors to the subprocess when you call exec*() using a magic undocumented mailbox (like UNIX pipes).

It would *definitely* make sense for VMS to provide a posix_spawn() API since it would be more efficient than the current method. Getting back to shells, I discovered bash had so many global variables that I needed to put them somewhere if I was going to try my plan to get the VMS port working more reliably by passing subshell data instead of bash (and every other UNIX shell)'s behavior of using fork() and the subshell inheriting stuff.

Bash is too large of a program to refactor into proper C++ quickly, and dash (the default /bin/sh in Debian/Ubuntu) is faster than bash at present, but my theory is that setjmp/longjmp, which bash *also* uses extensively internally, as a poor substitute for C++ exceptions, is slowed down by using that method of error handling, rather than C++ exceptions (which I'm trying to swap in as a replacement).

My theory is Itanium in particular is bad at setjmp/longjmp because it has so much state to save/restore (giant register stack). I'm upgrading all the Gentoo Linux packages on my rx2620 now that I've removed the Radeon 7500 card that I'd added that was causing Linux to freeze (apparently the Radeon "drm" driver doesn't like 2 cards in one Itanium), so I can run benchmarks there and also on the OpenVMS side.

I've decided to take a slightly different route for populating my preferred VMS POSIX environment, which is to start with a subset of the NetBSD userland. That's what MINIX did, and it worked out fine for them. Once I have something that looks acceptable to pkgsrc, as well as NetBSD's /bin/sh and UNIX utilities, BSD make, and C wrappers to make CC/CXX look like cc/c++, then I'll still only have C++03, but I'd also have a NetBSD userland at a fraction of the effort that porting GNU userland has turned out to be. I did learn more than I ever wanted to know about GNU autoconf, automake, libtool, and friends.

Cheers,
Jake Hamby