[Info-vax] VMS x86 performance ?

[abrsvc]

On Monday, 2 November 2020 12:50:07 UTC-5, geze... at rlgsc.com  wrote:
> On Sunday, November 1, 2020 at 6:52:12 PM UTC-5, Arne Vajhøj wrote:
> > On 11/1/2020 4:59 PM, clair... at vmssoftware.com wrote: 
> > > On Saturday, October 31, 2020 at 11:11:33 PM UTC-4, Michael Moroney wrote: 
> > >> "Craig A. Berry" <craig... at nospam.mac.com> writes: 
> > >>> On 10/30/20 3:43 PM, IanD wrote: 
> > >>>> The fact that it boots wickedly fast is also an indication is it 
> > >>>> not? 
> > >>> It's good news, but it may not mean what you think it means. I believe 
> > >>> they have done a lot of work on the boot path, so it's not necessarily 
> > >>> the same code that is running on other architectures. 
> > >> 
> > >> One big reason for the fast boot is the memory disk used, so that (other) boot drivers 
> > >> won't be needed. Load a chunk of memory once very early and run the rest of the early 
> > >> boot stuff from it. 
> > > 
> > > We understand the importance of performance comparisons but such 
> > > testing is not even on our radar yet. We have many, many more 
> > > important things to do in the next few months. As has been stated 
> > > previously, it would be a complete waste of time anyway until we can 
> > > turn on optimizing in the compilers. 
> > > 
> > > The plan has always been - keep adding more functions as we progress 
> > > through our V9.0 EAKs and eventually we will have V9.1, the complete 
> > > operating environment for an expanded Field Test. Moving from 9.1 to 
> > > 9.2 will be mostly performance and stability work. We now have 31 
> > > external users with access to the CrossTools Kit and the appliances 
> > > with V9.0-A, B, C, D, E, so we are making good progress. 
> > > 
> > > We had some nice breakthroughs in E. My KVM guest has now been up for 
> > > 31 days running a series of tests, usually 60 jobs on a single CPU. 
> > > Another more recent change regarding SMP now allows large batteries 
> > > of tests to run in 2P and 4P without issues. Coming in F will be full 
> > > SMP support, clustering and MSCP serving as well as other additions.
> > "Make It Work, Make It Right, Make It Fast" 
> > 
> > :-) 
> > 
> > Arne
> Arne,
> 
> I more or less agree with Clair. 
> 
> Generally speaking, benchmarking is hard. Computational benchmarks (e.e., non-privileged computations such as those used for MIPS and VUPS ratings) are one of the simplest species in that genus.
> 
> Benchmarking systems-level elements (e.g., context switches, system services, page faults) is complex and often difficult. I spent a good amount of time during my PhD research digging into these mechanisms in terms of the I/O system, and to say the problem is "non-trivial" is an understatement. Determining precisely what one is benchmarking is only the first small step. Separating the real performance information from the surrounding noise is a significant challenge. 
> 
> As Clair mentioned in his posting, the goal of these early versions is a working system. Accuracy first,  performance later.
> 
> - Bob Gezelter, http://www.rlgsc.com

I would agree, all to much focus is on hardware specifics or operating system function specifics.  I spent many years providing actual system performance benchmarks to customers using their own applications on real hardware being driven by "real" users (RTE testing actually) and the results were often eye opening.  Many clients thought that the "new" machine would improve throughput significantly only to realize that the limiting factor was not the system but the users themselves.

Testing system components has its place as long as the whole forest is examined and not just the trees.