[Info-vax] VUPS.COM relevance for modern CPUs

[Bob Gezelter]

On Friday, December 16, 2022 at 6:57:36 AM UTC-5, Mark Daniel wrote:
> Now, before everyone piles on, I understand the procedure provides an 
> indicative/comparative/finger-in-the-air measurement of the relative 
> performance of a VMS CPU relative to "the original VAX processor". 
> 
> It states as much in the prologue: 
> 
> |$! Provides an estimate of system CPU performance on OpenVMS systems. 
> |$! Use at your own risk. 
> 
> To clarify the platform of a particular run I added: 
> 
> |$ write sys$output f$fao("!AS with !UL CPU and !ULMB running VMS !AS",- 
> |f$edit(f$getsyi("hw_name"),"compress,trim"),- 
> |f$getsyi("availcpu_cnt"),- 
> |(f$getsyi("memsize")*(f$getsyi("page_size")/512)/2048),- 
> |f$edit(f$getsyi("version"),"collapse")) 
> 
> which provides the likes of: 
> 
> |HP rx2660 (1.40GHz/6.0MB) with 4 CPU and 14335MB running VMS V8.4-2L1 
> |Digital Personal WorkStation with 1 CPU and 1536MB running VMS V8.4-2L1 
> |innotek GmbH VirtualBox with 2 CPU and 7574MB running VMS V9.2 
> 
> It seems to be implemented as a tight DCL loop that executes almost 
> entirely in inner modes (I'm sure Brian can explain why). 
> 
> $ start_cputime = f$getjpi(0,"CPUTIM") 
> $ loop_index = 0 
> $ 10$: 
> $ loop_index = loop_index + 1 
> $ if loop_index .ne. init_loop_maximum then goto 10$ 
> $ end_cputime = f$getjpi(0,"CPUTIM") 
> 
> |Combined for 2 CPUs 0 50 100 150 200 
> | Interrupt State | | 
> | MP Synchronization | | 
> | Kernel Mode 21 |▒▒▒▒ | 
> | Executive Mode 21 |▒▒▒▒ | 
> | Supervisor Mode 58 |▒▒▒▒▒▒▒▒▒▒▒ | 
> | User Mode | | 
> | Compatibility Mode | | 
> | Idle Time 99 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ | 
> 
> 100% (or all-but) of this execution appears to be in inner modes. 
> Although X86 (above) seems to to have much more Kernel than other 
> architectures, e.g. IA64 below). There is no USER mode displayed in either. 
> 
> |Combined for 4 CPUs 0 100 200 300 400 
> | Interrupt State 1 | | 
> | MP Synchronization | | 
> | Kernel Mode 5 | | 
> | Executive Mode 18 |▒ | 
> | Supervisor Mode 78 |▒▒▒▒▒▒▒ | 
> | User Mode | | 
> | Not Available | | 
> | Idle Time 299 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ | 
> 
> There appear to be (at least) two versions of these procedures. The 
> later contains: 
> 
> |$! Modified: MAY-2010: Code updated by Volker Halle to address the 
> |$! following issues: 
> 
> and tweaks a few of the calculations. 
> 
> There also appear to be earlier tweaks allowing for Alpha processors 
> 
> |$ cpu_multiplier = 10 ! VAX = 10 - Alpha/AXP = 40 
> |$ cpu_round_add = 1 ! VAX = 1 - Alpha/AXP = 9 
>  th
> but none for Itanium. 
> 
> Are the Alpha tweaks sufficient to allow relevance for all 64bit CPUs? 
> 
> Are further tweaks required to make measurements on Itania relevant? 
> 
> And of course the same question for the successor to all three 
> architectures? 
> 
> -- 
> Anyone, who using social-media, forms an opinion regarding anything 
> other than the relative cuteness of this or that puppy-dog, needs 
> seriously to examine their critical thinking.

Been a bit busy the past few weeks with various things.

The best quote on benchmarks has been the US Environmental Protection Agency's disclaimer on it's automobile dynamometer-based fuel economy ratings "Your mileage may vary", generally rendered as "YMMV".

When CPU, and for that matter, mass storage devices were simple devices, without pipelines, caches, and the like, one could do simple benchmarks and obtain a useful result.

As far back as the late 1970s, pipelines and caches created a benchmark terrain full of cliffs, sinkholes, and plateaus. Back then, my research team saw benchmarks of the CDC 6600 vs the IBM System/370 Model 168 Submodel 3. Depewaysnding on the benchmark the comparison was a factor of 300%;  both ways. In other words, essentially an order of magnitude range.

Toss in three levels of CPU/memory caches, some of which are shared; virtualized mass storage at various levels; and other factors. The sum is that one is that getting a raw benchmark is only the beginning of the journey. Tuning can, and often does, have a very large field to explore.

- Bob Gezelter, http://www.rlgsc.com