[Info-vax] OpenVMS servers and clusters as a cloud service

[Stephen Hoffman]

On 2018-01-08 08:25:22 +0000, IanD said:

> I think VMS clusters have been bypassed now
> 
> Instead of building robust clusters with shared architecture, the IT 
> world built a multi threaded application layer instead
> 
> Hadoop mitigates the need for robust clusters. Just fire up a stack of 
> cheap Linux servers, deploy your workload across it, add however much 
> redundancy you want to handle for failures among the way and your done
> 
> VMS clusters are nice but who cares if you serve your disk across 6 
> nodes simultaneously when your workload is fixed to a particular node 
> anyhow. I'd that node goes down, your screwed, you've lost all that 
> compute time!
> 
> The world's moved beyond robust hardware, what people want is robust 
> applications and non stop workload processing
> 
> VMware captured the virilization market because it allowed people to 
> mitigates harder failure
> 
> The world is crying out for nonstop applications. Migrating whole 
> machines while it runs on VMware is one thing, having a workload that's 
> not stop that can be migrated to other machines and/or split and 
> combined while it runs I think will have business bearing down your 
> door with money in hand
> 
> What good is it if I run a batch job for 36 hours on VMS if the node 
> crashes and I have to start again. I've wasted 36 hours!
> 
> Find a solution to this problem and I think you find a pot of gold
> 
> Virtual processes / virtual workloads surely must be the next evolution 
> on from clusters?

As will undoubtedly be mentioned by various other folks, that's all 
entirely possible on OpenVMS from first principles or some such ilk.  
For one of the bits of API available to such apps, see the 
BATCH$RESTART mechanisms available in DCL.   For a different approach, 
use a transactional database.   App developers would often consider the 
use of DECdtm and RMS journaling here, too.

It's all also very close to what's been known as checkpoint-restart, 
toward process migration and the aforementioned VM guest migration, and 
to frameworks and mechanisms that can provide apps with consistent 
online backups.  General solutions to these requirements all currently 
requiring app-specific coding on OpenVMS.  Better abstractions and APIs 
for any of these could be added and then adopted by the apps and tools 
that needed the ability to recover from crashes, of course.  Akin to 
what's possible now, without using the lower-level and more complex 
APIs that are already available.  Easier and more consistent, and 
preferably better integrated with BACKUP and other OpenVMS tools and 
utilities.

App sharding and app consolidation and online backups and 
crash-recovery are not things that OpenVMS is currently built to do.  
That's all left to the app developers and to the third-party databases 
and other add-on dependencies the developers accrete.   There's not 
much documentation on how to achieve any this on OpenVMS, either.   
Combining failover and load balancing and rolling upgrades and the 
rest.  More than a little of the documentation and design focus has 
seemingly been around the uptime, with somewhat less around 
availability around outages and crashes and upgrades and patches.  
Around what happens when the uptime counters reset, and the counters 
inevitably will reset.

Yes; given Hadoop and other similar products, making uptime simpler and 
easier and cheaper to achieve is likely to get some attention from some 
ISVs.

As part of adding this stuff, I'd also want to know how often apps and 
systems crash, and that's metadata and telemetry that's not currently 
available to the folks at VSI, and it's only available for end-users 
and ISVs that have added uptime tracking.  Having written variations on 
those trackers for various folks, finding a boot record in the 
accounting file mean all of the existing process logins and batch 
startups have all been closed out, too.  But I digress.  In short, how 
often are apps and servers actually crashing and how big an issue is 
this for folks, and all of which plays into how willing the ISVs 
end-user developers will be amenable toward making the necessary 
modifications to apps and to system management practices.

But it's the existing OpenVMS developers that are the market for now, 
and VSI is busy catering to the existing ISVs and sites with the port 
and all the rest that's on the list.   Getting file system support for 
SSDs past 2 TiB, for instance.

ps: are there "vaxinations" necessary when visiting the "virilization market"?


-- 
Pure Personal Opinion | HoffmanLabs LLC