[Info-vax] Distributed Applications, Hashgraph, Automation

[Jim Johnson]

On Sunday, February 18, 2018 at 11:09:27 AM UTC-8, Stephen Hoffman wrote:
> On 2018-02-18 17:59:21 +0000, Jim Johnson said:
> 
> > Briefly, what I've seen about the cloud has many aspects, only a few 
> > align with outsourcing.  Not sure how much to go into that.
> > 
> > I was following the discussion on shared-everything vs. shared-nothing 
> > structures.  I've used both.  The RMS cache management was pretty 
> > expensive to run when I last looked at it.  It was especially bad for 
> > high write, high collision rate files.  This drove a different approach 
> > with the TP server in DECdtm.  It is structurally a shared nothing 
> > service on top of a shared everything access substrate.  It uses an 
> > unconventional leader election, in that the 'home' system always is the 
> > leader if it is alive, and the other systems elect one of themselves as 
> > the leader if it isn't.
> > 
> > (This was done via a pattern of use with the DLM, and I agree with 
> > Steve that either documenting the known patterns or encapsulating them 
> > for easier consumption could be useful)
> > 
> > This produced much better write performance, and good recovery 
> > availability times.
> 
> That experience is typical.  I've ended up splitting more than a few 
> apps similarly.   Either at the volume level, or within the app.   
> While SSDs have helped substantially with I/O performance, the 
> coordination involved with distributed shared writes ends up limited by 
> how fast you can fling lock requests around.    The byte-addressable 
> non-volatile storage that's coming on-line right now will only increase 
> the coordination load and the likelihood that sharding will be 
> considered or required, if you really want to use that memory at speed. 

Yup.  It wasn't just the cost of flinging the lock requests around.  The old RMS code, at least, propagated writes via the disk, rather than forking the I/O and sending a copy memory-to-memory.  From what I remember, we looked at doing that, but that was complex to get right didn't happen while I involved.  Maybe it has happened since.

One thing to highlight above is that the DLM mechanism we used produced an affinity aware election -- something that is uncommon today.  One reason it is uncommon, of course, is that we had a static affinity requirement and the research today is mostly focused on deriving unknown and evolving affinity requirements to drive leader election.  But still, for VMS clusters it is pretty straightforward to have failover for recovery and failback on restart in order to keep the cross node traffic down.

>   Faster networks also cause problems: https://lwn.net/Articles/629155/
> 
> > Let me add a caveat on the above.  I've been away from VMS for >15 
> > years.  All my data on VMS is very old, and is likely very out of date.
> 
> You're still rather current then, with a few errata.  Various of the 
> spinlocks have been much better broken up and there've been 
> optimizations in the lock management communications implementation and 
> elsewhere, and there've been incremental increases to FC HBA speeds and 
> NIC speeds, but there've not been significant changes to clustering nor 
> to DECdtm and the DLM since the XA-era work, and the cluster 
> configuration and management user interface is more or less the same 
> though has backslid somewhat in various areas.  Features such as SDN, 
> SMB, iSCSI, USB 3.1, UTF-8, and 40 GbE haven't yet arrived in OpenVMS.  
>  Some are underway and some are planned: 
> https://www.vmssoftware.com/products_roadmap.html

Hmm.  Ok.

I think I agree with a number of your comments then.  I especially share the concerns around deployment agility and scale.

To be clear, I also think it is great that VSI has taken the baton and pushing on this.  There is a lot of good core code to work from, and some excellent engineering principles behind the system.  I am still proud of the engineering in that system to this day.

Jim.
(also personal opinion :))
> 
> 
> -- 
> Pure Personal Opinion | HoffmanLabs LLC