[Info-vax] Do any disks still lie about writing data to permanent storage ?

[kemain.nospam at gmail.com]

>-----Original Message-----
>From: Info-vax <info-vax-bounces at rbnsn.com> On Behalf Of Jan-Erik
>Söderholm via Info-vax
>Sent: July-06-21 8:38 PM
>To: info-vax at rbnsn.com
>Cc: Jan-Erik Söderholm <jan-erik.soderholm at telia.com>
>Subject: Re: [Info-vax] Do any disks still lie about writing data to
permanent
>storage ?
>
>Den 2021-07-06 kl. 20:11, skrev Simon Clubley:
>> On 2021-07-06, Jan-Erik Söderholm <jan-erik.soderholm at telia.com> wrote:
>>> Den 2021-07-06 kl. 15:04, skrev Simon Clubley:
>>>> Talking about disks has reminded me of something else.
>>>>
>>>> In the not too distant past, some disks lied about having written
>>>> data to permanent storage instead of merely to some internal cache
>>>> that would be lost on a power failure.
>>>>
>>>
>>> They did not lie, of course. They said that the data was in the
>>> cache, which was perfectly true.
>>>
>>
>> We are not talking about SAN systems, we were talking about disk
>> drives such as Sata/SCSI/IDE disks. It's clear you have no experience
>> working with disk drives at the hardware level or you would have
>> recognised the issue I am raising.
>
>BS! I have worked with PDP-11/VAX/Alpha system from the 80s and forward
>incl MV-II, MicroVAX and up from that. And now including large IBM SAN
>systems. Using all sorts of different physical disk drives.
>
>>
>> Those disks have APIs built into them that tells the operating system
>> when the data in question has been committed to permanent storage by
>> the disk itself.
>>
>> Unfortunately, in the past some disks have lied about this and have
>> told the operating system that the data has been committed to
>> permanent storage when in fact it is still in a cache on the disk.
>
>That was not a "lie", it did so by design. And in many cases (or at least
>some) you could configure the drive to not do so, if you wanted.
>
>>
>> That means if one of these disks suffers a power failure before the
>> data _is_ written to permanent storage, then that data is lost. This
>> is after the operating system has been told the data is safe and has
>> potentially done other things based on that. :-(
>
>*IF* this has been a real issue, well, it had been known as a real issue.
>And if you concidered this to be a real issue, you selected your disk
hardware
>accordingly.
>
>>
>>>> Do any disks still lie about having committed data to permanent
>>>> storage or have we moved past that ?
>>>
>>> I would expect any modern SAN system to have extensive caching to
>>> reach good performance. Adn that the I/O finish when the data is in
>>> the cache.
>>>
>>> And why would the cache be lost at power failure?
>>>
>>> I think that your way of putting your question is "questionable".
>>>
>>
>> Next time Jan-Erik, before saying my question is "questionable", you
>> may want to ask yourself if in fact I am asking about something that
>> you are unaware of as is clearly the case here.
>
>BS! What I'm saying is that your are asking basied questions by calling
>something a "lie", that might just as well be well known design decisions.
And
>yes, this was well know from 80s and forward.
>Anyone working with disk drives knew this and made their own decissions.
>
>I certenly know what you are asking about.
>
>It is still not a "lie", which is what I'm pointing out. It was just the
way that
>(some) disk drives worked (and maybe works).
>
>But you asked about today. I would not concider this an issue today, since
>most storage is not based on single physical disks, at least not in storage
used
>from VMS systems of today.
>
>Lying is something a human does, not some bit of hardware.
>

As Jan-Erik correctly stated, there are no "lies".

The facts are that regardless of the physical server/storage device, there
are different write strategies that one can choose to implement based on
performance and data integrity requirements. This is an age-old design
decision that goes back decades.

"Write back" strategies are based on data being written to cache (HW or SW)
and then acknowledging the write is complete. Data is written to persistent
storage in the background. This results in higher performance, but lower
data integrity systems. This strategy can be somewhat mitigated with battery
backups on the cache systems. With new disk sizes in multiple TB's, this
write-back strategy can cause issues with the time to do file system
consistency checks after system crashes or power failures.

"Write through" strategies are based on data being written to persistent
storage before acknowledging the write is complete. These are slower
performing systems but higher data integrity file systems. 

Most storage controllers today (and many from decades ago) offer design
options to implement one write strategy or the other.

By default, the OpenVMS file system is based on an overall write through
strategy.

In case anyone wants to understand the huge difference in latency between a
cache write vs. a disk IO write vs. a network IO, check out this link:
<https://gist.github.com/hellerbarde/2843375>

On a semi-related note, while it has not really picked up steam to much,
Intel and other vendors have released new low latency solid state memory
solutions that emulate storage i.e. in addition to high performance, low
latency, they also have data persistence. This means the in-memory data is
protected after power failures.

Reference:
<https://www.intel.ca/content/www/ca/en/support/articles/000024018/memory-an
d-storage/intel-optane-memory.html>

Write through vs. write-back and overall data persistence is just one of
many design decisions that one should take into consideration.


Regards,

Kerry Main
Kerry dot main at starkgaming dot com










-- 
This email has been checked for viruses by AVG.
https://www.avg.com