[Info-vax] LLVM, volatile and async VMS I/O and system calls

[Stephen Hoffman]

On 2021-09-24 01:29:32 +0000, Lawrence D’Oliveiro said:

> On Friday, September 24, 2021 at 5:58:10 AM UTC+12, Simon Clubley wrote:
>> Are you familiar with true asynchronous I/O in normal applications 
>> where the operating system can write directly into your address space 
>> without the program having to wait around in a system call to actually 
>> receive that data and while the program is actually busy doing 
>> something else at the same time?
> 
> We normally do that with threading or shared memory these days.

At the level being discussed ($io_perform, $qio) all I/O is async by 
default.  The app queues the I/O request, and goes off to do... 
whatever.

OpenVMS apps can use the sync call format—which is a sync call wrapping 
around the underlying async call—and can then use ASTs or KP threads 
for multi-threading, or multiple processes.

The I/O eventually completes, or eventually fails, loads the I/O status 
block, and then triggers an event flag (often EFN$C_ENF "do not care") 
and the AST async notification. The IOSB is always set before the EF or 
the AST.

ASTs are in some ways a predecessor to a closure, and lack compiler 
support and syntactic sugar such as the block syntax found in clang, or 
the lambda syntax found in C++.

The DEC-traditional languages on OpenVMS sometimes have threading support.

(What I'm referring to as traditional: BASIC, Fortran, FORTRAN, Pascal, 
COBOL, BLISS, Macro32, etc. I'm here ignoring Java, Python, Lua, and 
whatnot, fine languages that those are.)

Older C (VAX C) had built-in parallelism support (which had some 
issues), and newer C has pthreads POSIX threading support.

On OpenVMS, pthreads are built on KP Threads.

Language-based async/await is not something that was common in years 
past, and the traditional OpenVMS compilers don't have support for 
that, nor for newer standards were those syntax features have been 
added.

Unix started out on a different path for I/O with largely sync calls 
for I/O, and developed async support later (epoll, kpoll, select, aio, 
pthreads, GSD, etc) to wrap around that.

select is a mess on OpenVMS, so we won't discuss that. aio and 
GSD/libdispatch don't exist on OpenVMS. etc.

Here, the usual OpenVMS app pattern would be an AST-based app, or maybe 
a threading app using pthreads or KP threads. The description I'd 
posted earlier in the COBOL thread is also somewhat GSD-ish, given its 
use of queues.

With ASTs, the app is either active in the mainline, or exactly one AST 
is active. Threads are somewhat more complex, and threads can and do 
operate entirely in parallel across multiple processors.

Both ASTs and threads require careful consideration of shared storage, 
which ties back to Simon's threads on compiler code optimization, as 
well as knowing the processor memory model.

Alpha in particular is very aggressive memory model, as compared to 
pretty much any other architecture available. And the COBOL thread 
involves Alpha.

It'd be possible to do all this in memory with a section and queues 
too, but that then means adding notifications (signals, DLM lock 
doorbells, $sigprc, etc) and eventually security and pretty soon most 
of the overhead of mailboxes or sockets.

Rolling your own communications interface is absolutely possible and 
was once fairly common. I've built and worked with more than a few 
communications APIs commonly using sections. Yes, pun fully intended.

For most cases with newer app development work or overhauls on OpenVMS, 
I'd tend to use sockets and not mailboxes (from over in the COBOL 
thread), but that's local preference. Sockets can let me move 
constituent apps further apart, should the app or server load 
increases. As has happened with apps I've worked on, the alternative 
tends to be mailboxes and sockets, which is more code and more 
complexity. And some have included section-based and driver-based 
comms. All that means more code, and more "fun" routing and logging and 
troubleshooting.

Creating an app that's basically one big ball of self-requeuing ASTs 
with a main that hibernates and wakes works pretty well for 
low-to-moderate-scale OpenVMS apps, too.




-- 
Pure Personal Opinion | HoffmanLabs LLC