[Info-vax] VUPS.COM relevance for modern CPUs

[Johnny Billquist]

On 2022-12-19 20:10, Simon Clubley wrote:
> On 2022-12-19, abrsvc <dansabrservices at yahoo.com> wrote:
>> On Monday, December 19, 2022 at 8:23:23 AM UTC-5, Simon Clubley wrote:
>>> On 2022-12-19, Mark Daniel <mark.... at wasd.vsm.com.au> wrote:
>>>>
>>>> With this in mind I knocked-up a small program to repeatedly call a
>>>> function using $CMEXEC which calls a function using $CMKRNL and that is
>>>> that. It measures how much effort is required compared to the simple
>>>> USER mode loop and reports it as b[ogo]VUPs.
>>>>
>>>> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>>>>
>>>> The real disappointment is my X86 VM. The rest of the results seem in
>>>> line with expectations.
>>>>
>>> What numbers did you see ?
>>>> PS. Looking for ideas, suggestions, criticism(s), etc. here...
>>>>
>>> The tests still "feel" rather artificial.
>>>
>>> My suggested alternative would be to write actual files away to disk
>>> using RMS sequential files and also indexed files. Maybe read them
>>> back as well.
>>>
>>> Repeat the sequential files test using direct QIO access and see what
>>> performance difference that gives when you bypass the transition to
>>> executive mode.
>>>
>>> (Yes, I know, RMS will add its own fixed overheads, but you will still
>>> be able to see a percentage difference across the various machines you
>>> are testing on and whether x86-64 VMS imposes a much higher performance
>>> overhead.)
>>>
>>> One obvious problem is that you will have to address issues round
>>> caching in your tests.
>>>
>> Why would you want I/O involved in a measurement of relative CPU power?  That makes no sense.  The VUPs rating has always been a relative CPU performance test.  You can argue whether User mode vs other modes makes sense.  I suppose that using Mark's updated program may make sense given that the newer versions of OpenVMS use software for some functions (modes really).  I don't believe that this is accurate as it will compare hardware vs software between a few models.
>>
> 
> Because Mark was reporting bad performance on x86-64 VMS compared to his
> other machines, but are the results artifically bad ?
> 
> We already know there is going to be an overhead because of the need to
> emulate executive and supervisor modes on x86-64 VMS, but Mark's results
> don't cover the elapsed time taken by doing "real" work while in executive
> mode.

The overhead would only be something that hits a little at mode 
switching. While just executing code, the cost of not having executive 
and supervisor in hardware should be zero.

And I agree that doing I/O usually means you won't see much of what the 
CPU performance actually looks like.

   Johnny