[Info-vax] Distributed Applications, Hashgraph, Automation

[Kerry Main]

> -----Original Message-----
> From: Info-vax [mailto:info-vax-bounces at rbnsn.com] On Behalf Of
> Stephen Hoffman via Info-vax
> Sent: February 21, 2018 1:35 PM
> To: info-vax at rbnsn.com
> Cc: Stephen Hoffman <seaohveh at hoffmanlabs.invalid>
> Subject: Re: [Info-vax] Distributed Applications, Hashgraph, Automation
> 
> On 2018-02-18 21:23:40 +0000, Jim Johnson said:
> 
> 
> > Fwiw, I spent the last 7 years (until I retired last month) working in
> > the Azure infrastructure.  It gave me a perspective on the cloud,
> > biased by being part of a cloud vendor.  So, this is just my personal
> > view on the cloud - one that I know is partial.
> >
> > The comparison to outsourcing is missing two aspects that are probably
> > most front and center to me.  First, it misses the dynamism about
> > resource acquisition and release.  An outsourcer, or any 'private
> > cloud' (inside company cloud) is not going to be able to provide the
> > ability to have quick peak and valley workloads with equally low cost.
> > The public clouds can.  That has led to workloads that are inherently
> > transient.  They spin up 100's to 1000's of VMs for a short period, use
> > them, and then give them back.  You need a lot of incoming workloads
> to
> > amortize that effectively.
> >
> > This also pushes on deployment and configuration agility.  If you're
> > expecting to use 1000 VMs for an hour, but you first have to spend an
> > hour deploying to them, you're not going to be happy.  So that drives
> > deployment times to small numbers of minutes.
> 
> Kerry's business is seemingly dead-center in this market, but Kerry's
> apparently not (yet?) interested in this.  Gaming has its hits and its
> misses, big launches and hopefully long tails, and system and network
> loads can vary widely.   It's also distributed, which gets into
> discussions of globally-distributed locality and which haven't been
> mentioned here yet.   Based on what's been presented and what's been
> posted here, Kerry is planning on over-provisioning and clustering
> private hardware.   Which also includes dealing with some of the
> difficulties within OpenVMS around preferably-unattended installations
> and configurations and lifecycle distributed security.  A large part of
> this implementation hasn't seen an overhaul in OpenVMS since ~V6.0
> with
> PCSI replacing VMSINSTAL.
> 

Large shops inevitably use custom methods to deploy "gold" images. 

A gold image puts all patches, agents, hardening in one custom image so you simply lay that image down on the HW, customize it for things like name, IP etc. and then boot it into the solution.

Reference:
<http://h41379.www4.hpe.com/openvms/journal/v15/blade_servers_ovms.pdf>

Also, reference the new Synergy HW based deployment option from HPE: (think OpenVMS X86-64)
<https://www.youtube.com/watch?v=eB59ycpv0w4> 

Re: public clouds
Just to clarify, public cloud may have some benefits, but for any environment where you need to tightly control overall solution latency, public cloud is not the answer.

As a reminder, much of the public Internet is based on MPLS protocols which gives one availability, but if something internal to the MPLS cloud is unavailable it will re-route the connection and you have no idea of what your new latency is going to be.

Reference:
<http://www.bbc.com/news/business-36854291>
"Onlive had many challenges. but perhaps the most difficult was latency, or time delay. It is the perceivable amount of time players wait for a game to send commands to a data centre and then send back the results."

> > This is where batch has gone, from what I can see.  Whether it is Azure
> > Batch, Hadoop, or something else.
> 
> DECscheduler was vastly easier than dealing with home-grown batch
> procedures twenty years ago, and options and alternatives have only
> gotten better.  OpenVMS doesn't even have something of the sheer
> sophistication of cron, which is not competitive.
> 

Again, most enterprise Customers would rather purchase a cross platform scheduler with a common look-n-feel than have a whizz bang scheduler on one platform that does not run on all of its other platforms.

> > But aren't all my workloads basal (i.e. always must be there)?  Maybe
> > today.  I've watched a lot of basal workloads turn into transient
> > workloads as people have understood that there's value in doing so.  It
> > wasn't that they had to be basal, just that it was easier to express if
> > there was no real transient resource capability.  There are indeed
> > basal workloads, but they're typically a smaller subset than people
> > first expect.
> >
> >
> > The second aspect also has to do with agility.  Again, my understanding
> > is from thinking about software providers.  Every vendor is in a
> > repeated contest with their competitors.
> 
> The better ones are in a contest with themselves; with replacing and
> updating their own products.
> 
> > This means that speed of getting from requirement to product in front
> > of the potential customer matters -- i.e. the length of their relative
> > release cycles.  A shorter release cycle matters - it lets you get
> > ahead of your competition, showing features that are more relevantd
> and
> > appearing more responsive (note that this doesn't say that the
> > engineers aren't working on about the same cool features in both
> > places, only that the potential customer is not seeing them for the
> > company with the longer release cycle).
> >
> > And one aspect of the cycle time is the cost of release.  The higher
> > the cost, the more work that has to go into the release for it to be
> > justified.  For a traditional ('box') product, this is rarely shorter
> > than 6 months.
> 
> VSI hasn't embraced continuous releases, though they'll likely be
> thinking about that.  Outside of patches.
> 
> > These are just things that have been true.
> >
> > The cloud disrupted this in a big way.  The delivery mechanism and the
> > structure of most services (including the incorporation of devops) has
> > driven this cycle time to as low as minutes for higher level features
> > to a few months, worst case, for deep technical changes.  That means
> > that a cloud service competing with a box product is always ahead, and
> > often way ahead, of responding to changing customer requirements.
> >
> > Note that this is a lot more than just changing the delivery channel.
> > That is part of it.  But it also requires care on the engineering
> > processes, on the service architecture, on monitoring and telemetry,
> > and inclusion of devops.  For the last, I had a continuous love-hate
> > relationship with devops - I loved the insight it gave me on my
> > customers and how my service actually worked, and hated the 2AM
> > calls. 😃
> >
> > This is incomplete - it is just two top level thoughts that I had when
> > I was reading.  I honestly don't know how much of this is relevant to
> > VMS.  I'm just sharing my thoughts.  YMMV.  Again, just my personal
> > opinions.
> 
> It's what I've seen from the customer side of this mess, too.
> Duplicating what already exists is not competitive, and it increases
> costs and testing efforts, or makes the release cycles slower, or both.
>  I really need a good reason to do that.
> 
> Beyond discussions of distributed operations and clustering — what
> kicked the whole discussion off here – OpenVMS really needs to be
> better around installing into a hosted environment, even if some of the
> folks — not the least of whom is Kerry — don't plan on ever using that
> approach.  Even if it's updates and changes for easier installations
> and configurations and operations in a private cloud sans hardware
> console, for those that have reasons to (try to) own the whole stack.
> 

See deployment links above.

Could OpenVMS installations be better from a single server install perspective? Absolutely.

However, as noted above, large enterprises do not use the traditional install processes (on any platform).

Once OpenVMS is available in a virtual KVM environment, I would anticipate things similar to VMware's templates will be available for OpenVMS.

Regards,

Kerry Main
Kerry dot main at starkgaming dot com