[Info-vax] Synchronization, processes, threads, XQP & ACPs (was: Re: Oracle Database vs Oracle/Rdb

[Stephen Hoffman]

On 2019-06-28 02:07:47 +0000, Dave Froble said:

> Now today, with lots of CPUs (cores), multiple things can occur 
> simultaneously, and if every user is waiting on the Oracle engine to 
> service their needs, it probably is harder to scale that up, than to 
> scale up with the Rdb concept of running in the user process.

I'm going to assume that that was meant to be "the XQP concept of 
running in the user process", and not "the Rdb concept of...".

Is threading more complicated than single-threaded, or single-threaded 
with ASTs?  Sure.  Yes.  But dividing up the work across 
multi-threading and across multi-processing is where better scaling is 
available, per Amdahl. One core is only one core, whether it's for an 
app or for the activity expected of the so-called primary processor 
within the OpenVMS symmetric multi-processing SMP design.

The XQP provides the low-level parts of ODS-2 and ODS-5 volume 
structure, upon which RMS-based apps and most-but-not-all non-RMS-based 
apps operate.  There are other ACPs used on OpenVMS that provide 
similar functions, though for other volume structures.  The F11XQP XQP 
replaced the F11BACP ACP some years ago and by mapping parts of the ACP 
processing into each process.  AFAIK, it is still possible to summon 
the use of F11BACP, if you're so inclined.

There's not a whole lot of difference here though, in terms of the 
synchronization requirements.  The same-process construct used by XQP 
does avoid a process context swap that was once endemic with the ACP it 
replaced design, and avoids the interprocess communications.

Now the F11BACP wasn't multi-threaded, while Oracle Rdb is.  Though per 
one of the Oracle folks, they're seemingly using their own threading 
package, and not OpenVMS KP Threads.
https://pdfs.semanticscholar.org/eb8b/db98231f478e200e00ed9f70d1b125f9f2cb.pdf

But in terms of the required coordination, it's all pretty much the 
same whether running across processes or in the same process context or 
potentially across threads. There still has to be coordination with the 
other processes and other threads that might be sharing access.  
Different API to get the data from the client app and to the XQP and 
back again, but otherwise still providing what the ACP necessarily 
provided.

As for the savings around context switches?  With the increasing 
numbers of cores, busy processes can remain active in parallel, with 
fewer process context swaps necessary.  The old VAX and Alpha one-core 
server designs were swapping all the time, yes.  With 16- and 32-core 
server configurations becoming commonplace, a whole pile of processes 
and a whole pile of threads can all be running in parallel.  As for 
where we're headed, rumors of 64-core AMD Ryzen Threadripper processor 
designs have been rife recently, too. All in a one-socket 
probably-going-to-be-NUMA design.

As for the overhead of interprocess communications, that's still 
comparatively poor on OpenVMS and I don't know of any substantial 
changes in that area in the past decade or two, though some other 
systems such as the L4 family have been working to optimize that and 
have greatly reduced the associated overhead of messaging.  L4 tosses 
around a whole lot of messages, too.  
https://ts.data61.csiro.au/publications/nicta_full_text/8988.pdf

Threading on OpenVMS—where most designs are headed when AST-based 
designs aren't getting the required throughput—is not that far off of 
multi-processing.  The process scheduler is involved with threads and 
hyper-threads and processes, and with a shared address space for the 
threads within a process.  The XQP is capable of operating across 
multiple threads, but is not itself multi-threaded.   Then there's the 
discussion of load balancing and scaling, and of having larger tracts 
of code all mapped into the same address space as user code and thus 
into a privileged mode/ring, and of ensuring the isolation of all of 
that privileged code sharing the address space with untrusted code.  
Whether against crashes or corruptions, or against more explicit 
shenanigans.  And somewhat less often, discussions around performing 
rolling upgrades and fault tolerance of the code involved.

Now debugging the XQP is... fun.  This as it's mapped into the same 
process, it's in inner-mode, and it's what you're using for your 
process I/O.  Which means it'll be preferable to debug in a different 
process, or remotely across OpenVMS systems. Debugging code that is 
operating in user mode / ring 3 is preferred to debugging code 
operating in a privileged mode / ring.

We're also starting to see restrictions by some operating system 
vendors around privileged-mode application code, too. And increasingly 
seeing code-signing requirements, too.  Privileged mode code / 
inner-ring code is inherently fully privileged, whether that code is 
DCL or some DCL extension, or a file system not in user space non-FUSE 
design, privileged shareable image or execlet, or otherwise.  OpenVMS 
trails what is becoming common here.

There are trade-offs between single-address-space kernel 
designs—whether monolithic kernels, loadable kernels, and otherwise—and 
microkernel designs.  Just as there are differences and trade-offs 
among unthreaded, AST-based, and multi-threaded application code.

TANSTAAFL.

ps: As for apps that can or that want to run "everything database" in 
one process, SQLite might be interesting.  For those folks that want or 
need to spread app processing acround and across processes and 
processors, maybe Erlang.  Etc.






-- 
Pure Personal Opinion | HoffmanLabs LLC