[Info-vax] Distributed Applications, Hashgraph, Automation

[DaveFroble]

Stephen Hoffman wrote:
> On 2018-02-18 21:23:40 +0000, Jim Johnson said:
> 
> 
>> Fwiw, I spent the last 7 years (until I retired last month) working in 
>> the Azure infrastructure.  It gave me a perspective on the cloud, 
>> biased by being part of a cloud vendor.  So, this is just my personal 
>> view on the cloud - one that I know is partial.
>>
>> The comparison to outsourcing is missing two aspects that are probably 
>> most front and center to me.  First, it misses the dynamism about 
>> resource acquisition and release.  An outsourcer, or any 'private 
>> cloud' (inside company cloud) is not going to be able to provide the 
>> ability to have quick peak and valley workloads with equally low 
>> cost.  The public clouds can.  That has led to workloads that are 
>> inherently transient.  They spin up 100's to 1000's of VMs for a short 
>> period, use them, and then give them back.  You need a lot of incoming 
>> workloads to amortize that effectively.
>>
>> This also pushes on deployment and configuration agility.  If you're 
>> expecting to use 1000 VMs for an hour, but you first have to spend an 
>> hour deploying to them, you're not going to be happy.  So that drives 
>> deployment times to small numbers of minutes.
> 
> Kerry's business is seemingly dead-center in this market, but Kerry's 
> apparently not (yet?) interested in this.  Gaming has its hits and its 
> misses, big launches and hopefully long tails, and system and network 
> loads can vary widely.   It's also distributed, which gets into 
> discussions of globally-distributed locality and which haven't been 
> mentioned here yet.   Based on what's been presented and what's been 
> posted here, Kerry is planning on over-provisioning and clustering 
> private hardware.   Which also includes dealing with some of the 
> difficulties within OpenVMS around preferably-unattended installations 
> and configurations and lifecycle distributed security.  A large part of 
> this implementation hasn't seen an overhaul in OpenVMS since ~V6.0 with 
> PCSI replacing VMSINSTAL.
> 
>> This is where batch has gone, from what I can see.  Whether it is 
>> Azure Batch, Hadoop, or something else.
> 
> DECscheduler was vastly easier than dealing with home-grown batch 
> procedures twenty years ago, and options and alternatives have only 
> gotten better.  OpenVMS doesn't even have something of the sheer 
> sophistication of cron, which is not competitive.
> 
>> But aren't all my workloads basal (i.e. always must be there)?  Maybe 
>> today.  I've watched a lot of basal workloads turn into transient 
>> workloads as people have understood that there's value in doing so.  
>> It wasn't that they had to be basal, just that it was easier to 
>> express if there was no real transient resource capability.  There are 
>> indeed basal workloads, but they're typically a smaller subset than 
>> people first expect.
>>
>>
>> The second aspect also has to do with agility.  Again, my 
>> understanding is from thinking about software providers.  Every vendor 
>> is in a repeated contest with their competitors.
> 
> The better ones are in a contest with themselves; with replacing and 
> updating their own products.
> 
>> This means that speed of getting from requirement to product in front 
>> of the potential customer matters -- i.e. the length of their relative 
>> release cycles.  A shorter release cycle matters - it lets you get 
>> ahead of your competition, showing features that are more relevant and 
>> appearing more responsive (note that this doesn't say that the 
>> engineers aren't working on about the same cool features in both 
>> places, only that the potential customer is not seeing them for the 
>> company with the longer release cycle).
>>
>> And one aspect of the cycle time is the cost of release.  The higher 
>> the cost, the more work that has to go into the release for it to be 
>> justified.  For a traditional ('box') product, this is rarely shorter 
>> than 6 months.
> 
> VSI hasn't embraced continuous releases, though they'll likely be 
> thinking about that.  Outside of patches.
> 
>> These are just things that have been true.
>>
>> The cloud disrupted this in a big way.  The delivery mechanism and the 
>> structure of most services (including the incorporation of devops) has 
>> driven this cycle time to as low as minutes for higher level features 
>> to a few months, worst case, for deep technical changes.  That means 
>> that a cloud service competing with a box product is always ahead, and 
>> often way ahead, of responding to changing customer requirements.
>>
>> Note that this is a lot more than just changing the delivery channel.  
>> That is part of it.  But it also requires care on the engineering 
>> processes, on the service architecture, on monitoring and telemetry, 
>> and inclusion of devops.  For the last, I had a continuous love-hate 
>> relationship with devops - I loved the insight it gave me on my 
>> customers and how my service actually worked, and hated the 2AM calls. 😃
>>
>> This is incomplete - it is just two top level thoughts that I had when 
>> I was reading.  I honestly don't know how much of this is relevant to 
>> VMS.  I'm just sharing my thoughts.  YMMV.  Again, just my personal 
>> opinions.
> 
> It's what I've seen from the customer side of this mess, too.   
> Duplicating what already exists is not competitive, and it increases 
> costs and testing efforts, or makes the release cycles slower, or both. 
> I really need a good reason to do that.
> 
> Beyond discussions of distributed operations and clustering — what 
> kicked the whole discussion off here – OpenVMS really needs to be better 
> around installing into a hosted environment, even if some of the folks — 
> not the least of whom is Kerry — don't plan on ever using that 
> approach.  Even if it's updates and changes for easier installations and 
> configurations and operations in a private cloud sans hardware console, 
> for those that have reasons to (try to) own the whole stack.

As many are aware, I don't get out much, so I have no idea what percentage of 
users would fit the description of needing varying resources.  My experience is 
more with situations where the requirements are more fixed, basically the same 
every day.

Got any numbers showing the distribution of users based upon varying, or 
non-varying requirements?


-- 
David Froble                       Tel: 724-529-0450
Dave Froble Enterprises, Inc.      E-Mail: davef at tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA  15486