[Info-vax] Another "automatic update"

[Arne Vajhøj]

On 3/27/2019 12:08 AM, Dave Froble wrote:
> Got a bit busy today.  Quite interesting, and disgusting.
> 
> A networking task that has been working very well for months, if not 
> years, got an error and aborted.  Actually, did it twice.
> 
> This happened right after an automatic update of WEENDOZE server 2012, 
> and of course, it's not a consistent error, just intermittent.  Oh, joy!
> 
> Turns out it's an error from WEENDOZE TCP/IP, where it attempts to 
> connect to another connection request while the listener is currently 
> attached to another connection.
> 
> Ok, nothing in the application program can affect this.  Best I can 
> determine is it's probably something introduced by the automatic update.
> 
> So, now, where were all those arguments for just rolling in updates 
> without some serious testing?  So we have this brave new world of 
> constant automatic updates.  Be afraid.  Be very afraid.
> 
> To be fair, since it's intermittent, testing just might not have caught it.
> 
> Moved the task over to a WEENDOZE 7 system and it's been working without 
> any problems.  But, that's putting off the issue, not solving it.

When security patches come out you have some options:
* put it on quickly with minimal smoke test
* test it thoroughly and put it in a scheduled batch
* ignore it

If you put it on then there is definitely a risk that
the patch could cause problems.

If you don't put it on then there is the risks that
the vulnerability could be used for bad purposes:
* take down the system
* steal data from the system
* modify data in the system

Your choice would depend on the context:
A) what is the risk of a problem with the patch
B) what would be the cost of a problem with the patch
C) how big is the risk of the vulnerability being
    exploited
D) what is the potential cost of the vulnerability being
    exploited

re A)

We hear about such problems frequently - several times per year.
But there are thousands of patches rolled out to millions of
servers every year. The statistical chance of a problem
must be pretty small.

re B)

With a good restore process in place it should only
cost a few hours of downtime. Of course that can also
be quite expensive for some companies.

re C)

Very system specific.

re D)

That cost is typical pretty big. The business loss
if a companies data get stolen can be huge. Equifax
said their data breach costed 430 M$ in direct cost -
on top of that can come long term loss due to bad
reputation. The business impact of data being modified
could be even bigger. A manufacturing system modified
to produce defect output or a financial system
with money moved around unauthorized - that would likely
mean the end of the company.

So unless #C is very small, then most companies will
patch because #D is much bigger than #B.

Arne