[Info-vax] naming convention in VMS MAIL

[Stephen Hoffman]

On 2018-12-20 01:29:42 +0000, Dave Froble said:

> And some of us are rather happy with that.  When I do a write, I really 
> want to believe that the data is truly written.

I don't doubt it.

Some data needs to be written to persistent storage before completion 
is returned.  Transaction logs, for instance.  Other data can be more 
aggressively cached.  More than a little should preferably be written 
as a group of I/O related operations, or not at all.  More than a 
little of what OpenVMS apps do is transactional, but fewer apps are 
written that way. Some data written need not be written to persistent 
storage at all.  With transaction logs in persistent storage, running 
from cache becomes tractable.

Ponder writing your transactional log to NVMe or persistent memory, and 
flushing that out to SSD and eventually HDD archival storage, yet 
running live operations out of DRAM.  So long as those transaction logs 
are available, the DRAM can be rebuilt at reboot.  And if the power 
holds, DRAM contents can be re-loaded from backing storage.

Put a different way, different processor and storage speeds and 
capacities can lead to different designs.  OpenVMS is not very good at 
presenting this, as the APIs and the app designs are still heavily 
focused at VAX-era records read and written from VAX-era HDD storage.

As an example, I was working on some data file processing a while back, 
and realized it was vastly faster to read the whole file into virtual 
memory and process it there, rather than slinging single- or 
several-sector I/O at storage.  Fewer and bigger I/O requests still win 
over more and smaller I/O requests, and the application code is much 
simpler.

As a different example, there are various apps that 
are—unfortunately—I/O bound for passing around counters and analogous.  
These "critical" files can be redesigned and rebuilt certainly, but the 
apps can end up saturating parts of the I/O subsystem with their 
designs.  OpenVMS doesn't have an easy way to tell RMS and the XQP that 
the entire $open/$write/$read/$close sequences can be cached and don't 
need to be flushed.  As apps are scaled, these sorts of surprises can 
appear.

Yes, SSDs have ameliorated many previously-problematic app designs, and 
NVMe storage and persistent memory will shift yet more designs and 
design trade-offs.  The hardware has been changing faster than many of 
the existing OpenVMS app designs.

> One can mention batteries, UPS, and such, but, that doesn't give me the 
> same warm fuzzy feeling.

Or RAID controller caches.  Those RAID controller caches are thankfully 
involving batteries rather less often, too.

And some servers now have embedded battery backup.  And I suspect 
nobody has looked at the powerfail-restart processing in OpenVMS in 
decades.

Etc.

> I seem to recall, sometime in the far past, a physics course and 
> learning about energy and work.  No such thing as a free lunch, 
> perpetual motion, and such.  If you wanted some amount of work done, 
> you had to provide the required energy.  Maybe that applies here.  If 
> you want to do the entire job, it takes required effort.  If you're 
> happy with half the job, then perhaps it might seem to be faster.

While I understand where you're headed with that, what I'm referencing 
is better efficiency around what I/O the apps do, and how tolerant apps 
are to partial transactions and partial writes, and where caching and 
larger writes can work better.

Non-trivial software apps are also certainly subject to their own form 
of entropy, but that's fodder for more than a few different discussions.

The design of the Mail storage database likely hasn't been looked at in 
decades, and what we have now has long been limited by its own history 
and implementation, and by the I/O and storage limitations of OpenVMS 
itself.

> That [Intel x86-64 embedded digital signal processor] was interesting.  
> The way I read it (remember, I don't get out much, and could be 
> mistaken) is Intel builds a back door for the hackers.  Is that right?

x86-64 is a massively complex device, irrespective of the vendor.  With 
sometimes substantial chip-to-chip variances.  Embedded assistance for 
manufacturing goes back decades, not the least of which involves JTAG 
diagnostic ports.  This is little different.




-- 
Pure Personal Opinion | HoffmanLabs LLC