[Info-vax] Returning data from Cobol AST routine.

[Jan-Erik Söderholm]

Den 2021-09-21 kl. 18:45, skrev Stephen Hoffman:
> On 2021-09-20 22:55:02 +0000, Jan-Erik Sderholm said:
> 
>> The idea is to have an AST routine that will read from a mailbox when 
>> something is written to it. My idea was that the read of the mailbox 
>> would be using an AST to avoid polling the mailbox.
> 
> In no particular order...
> 
> Start with chapters 6 and 8 in the first VSI Programming Concepts Manual. 
> Memory sync and AST operations details.
> 
> All data shared between mainline and AST must be atomic-accessed, or must 
> be protected by atomic operations, or some other locking.

I thought of not re-queing the QIO until the current data had been
processed. That way there will not be any new AST until the whole
appliation is ready for that, right?

> 
> The AST can arrive between pretty much any two hardware instructions within 
> any code sequence, or is best assumed to arrive at the least opportune 
> moment. Including within an unaligned variable read, or within a multi-part 
> update operation such as an un-interlocked queue update.
> 

My thought was to only queue an QIO when I know the application is ready.


> Interlocked bitflags or interlocked queues are commonly used to avoid these 
> races. There are other ways too, though these interlocked operations tend 
> to be fast and fairly lightweight. These were VAX hardware instructions, 
> and are now available as C built-ins and as LIBRTL calls.  Calls to $SETAST 
> can be used to brute-force block AST activity.
> 
> For cases processing async I/O such as this, an app design maintaining a 
> set of two or sometimes three queues is fairly common. The mainline 
> hibernates, if it doesn't have something to do. One queue pre-populated at 
> startup and used as a free list of data structures available for use, one 
> queue of data structures from mainline to AST, and one for structures AST 
> to mainline. The interlocked queues avoid issues with variable tearing, and 
> with race conditions. The structures can also contain the IOSB, the data 
> buffer, the interlocked queue headers, and whatever other data is required.
> 
> Make the number of buffers pre-allocated programmable, and add some 
> instrumentation for when the free list is exhausted. It's absolutely 
> possible to dynamically add some structures when the freel queue s empty, 
> at the cost of some added complexity.
> 
> Use two unidirectional mailboxes whenever mailbox message traffic is 
> bidirectional. One reader, one or more writers, for each of the two 
> mailboxes. Don't try to multiplex a mailbox. That way leads to gnarly.
> 
> Always have a scheduled timer AST or (locally preferable) have a scheduled 
> wakeup call operating in the background, triggering a routine that 
> processes the queues every ten or thirty or minute, on the off chance an 
> AST or event flag gets lost somewhere. This happens more often than any of 
> us would prefer, and the scheduled wakeup. This so that the AST issues a 
> $wake, and the mainline loops and does its processing and eventually issues 
> a $hiber when the work-pending queue is empty. Once started, the mainline 
> initializes the queues and queues the ASTs, then hibernates. The ASTs 
> arrive as the I/O completions arrive, and each can trigger a wakeup call 
> and can shuffle buffers among the interlocked queues and can also re-queue 
> the I/O operation for the next I/O operation. And the scheduled wakeup call 
> masks any lost wakeup calls, at minimal cost. (Wakes are coalesced into 
> one, and I've worked with a few app designs where a wake was occasionally 
> lost.)
> 
> For a few of these app cases I've used global sections for app control, 
> sometimes lock manager doorbell locks, sometimes controlled activity 
> through signals, sometimes a management mailbox pair, and it's often 
> possible to pass management commands into the app through the mailbox.
> 

Most of the above I do not understand. But keep it in mind if there
is some scenario where it seems to fit in.

> Protect the mailboxes against unauthorized access correctly, and there can 
> be protocol design cases where looking at the PID of the message sender can 
> be necessary.

All access to the mailbox (both "ends") are from our own code.

> 
> You will want to consider including some sort of a protocol version within 
> the mailbox message traffic too, as that can makes message protocol 
> upgrades a whole lot easier.

Good idea.

> 
> These app designs pretty quickly become state machines, too. It can be 
> easier to start out with interlocked queues and state machine than to try 
> to retrofit this. This having retrofit more than a few state machines into 
> apps that got way too gnarly.
> 
> If there's any chance the other end of the connection is untrusted, parsing 
> gets vastly more difficult.

All access to the mailbox (both "ends") are from our own code.

> 
> COBOL will be more involved, as it doesn't do pointers at all well, and the 
> designs I tend to use are rather more dynamic.
> 

Good for you. Our application build environment is Cobol based.