[Info-vax] VMS Software Q1 '23 Update

[Johnny Billquist]

On 2023-01-28 23:35, Arne Vajhøj wrote:
> On 1/28/2023 3:01 PM, Steven Schweda wrote:
>>> Consider the case where the numbers represent money.
>>
>>     Who cares what the numbers represent?  For example, if you round to
>> the nearest integer, then 1.49999 and 1.50001 differ by only 0.00002,
>> but one rounds to 1, and the other to 2, so the lame test says that
>> they're "unequal".  And 1.50001 and 2.49999 differ by 0.99998, but they
>> both round to 2, so the lame test says that they're "equal".  Any test
>> which treats a difference of 0.00002 as significant, and a difference of
>> 0.99998 as negligible, is, I claim, (obviously) a bad test, regardless
>> of what the numbers might represent.
> 
> There are some specific rules for money usually about
> always rounding 2 digits.
> 
> 1494 dollars divided by 1000 = 1.494 get rounded to 1.49
>                                diff 0.002          not equal
> 1496 dollars divided by 1000 = 1.496 get rounded to 1.50
> 
> 1496 dollars divided by 1000 = 1.496 get rounded to 1.50
>                                diff 0.008            equal
> 1504 dollars divided by 1000 = 1.504 get rounded to 1.50
> 
> Arne
> 
> PS: One should never use traditional binary based floating
>      point for storing money instead use some decimal based
>      type. But sometimes calculations switch to floating point
>      anyway.

So by your own example, two values differing by 0.002 could be 
considered not equal, but two values differing by 0.008 could be 
considered equal. Don't you see that this leads to a very bad algorithm 
for comparing for approximate equal?

More generally, 0.002 of difference might be considered equal or not 
equal purely based on what the actual values are. For some, they will be 
equal, and for some others not. The delta that is required for two 
numbers to be considered unequal varies based on what the actual numbers 
are.

At least in my book, that is a clear indication that it's a very bad method.

   Johnny