[Info-vax] VMS benchmarks

[Arne Vajhøj]

On 12/15/2023 10:21 AM, Arne Vajhøj wrote:
> On 12/6/2023 4:35 PM, Arne Vajhøj wrote:
>> https://www.vajhoej.dk/arne/articles/vmsbm1.html
> 
> I have updated with some Groovy numbers.
> 
> Rather impressive numbers.
> 
> Groovy out of the box is OK.
> 
> Groovy with the critical functions marked with @CompileStatic
> (requires all variables to have an explicit type) performs great.
> 
> I think it is a pretty cool feature to be able to write
> both traditional script style code without any explicit type
> information and put explicit type information and an
> annotation on a performance critical function to get
> compiled language performance.

Here are another test with some even more
impressive numbers.

And note this is stuff running on VMS x86-64 9.2-1 today.

C: 6.3 s, 6.3 s, 6.0 s, 4.1 s (depending on compiler and optimization)
Groovy script style : 51 s
Groovy with type declarations : 38 s
Groovy with type declarations and @CompileStatic : 3.5 s
Groovy calling Java code : 3.4 s

So one can write compact script code at a Python level
and get C performance of a performance critical function
by just declaring types explicit and put an annotation on.

Obviously do not conclude too much from a benchmark of
a function with one line of code.

Code and test output below.

Arne

$ type fib.c
#include <stdio.h>

#include "high_res_timer.h"

int fib(int n) {
     return n < 2 ? n : fib(n - 1) + fib(n - 2);
}

static const int REP = 3;

int main()
{
     printf("C:\n");
     for(int i = 0; i < REP; i++)
     {
         TIMECOUNT_T t0 = GET_TIMECOUNT;
         int ans = fib(45);
         TIMECOUNT_T t1 = GET_TIMECOUNT;
         printf("fib(45) = %d took %f seconds.\n", ans, (t1 - t0) * 1.0 
/ UNITS_PER_SECOND);
     }
     return 0;
}

$ cc fib
$ link fib
$ run fib
C:
fib(45) = 1134903170 took 6.289937 seconds.
fib(45) = 1134903170 took 6.349937 seconds.
fib(45) = 1134903170 took 6.329937 seconds.
$ cc/opt=level:5 fib
$ link fib
$ run fib
C:
fib(45) = 1134903170 took 6.489935 seconds.
fib(45) = 1134903170 took 6.299937 seconds.
fib(45) = 1134903170 took 6.349937 seconds.
$ clang fib.c
$ link fib
$ run fib
C:
fib(45) = 1134903170 took 6.039940 seconds.
fib(45) = 1134903170 took 5.999940 seconds.
fib(45) = 1134903170 took 5.959940 seconds.
$ clang "-O3" fib.c
$ link fib
$ run fib
C:
fib(45) = 1134903170 took 4.049959 seconds.
fib(45) = 1134903170 took 4.089959 seconds.
fib(45) = 1134903170 took 4.079959 seconds.
$ type Fib.groovy
def fib1(n) {
     return n < 2 ? n : fib1(n - 1) + fib1(n - 2)
}

int fib2(int n) {
     return n < 2 ? n : fib2(n - 1) + fib2(n - 2)
}

@groovy.transform.CompileStatic
int fib3(int n) {
     return n < 2 ? n : fib3(n - 1) + fib3(n - 2)
}

REP = 3

def test(lbl, f) {
     println("${lbl}:")
     for(i in 1..REP) {
         t0 = System.currentTimeMillis()
         ans = f(45)
         t1 = System.currentTimeMillis()
         println("fib(45) = ${ans} took ${(t1 - t0) / 1000.0} seconds.")
     }
}

test("Groovy Python style", { n -> fib1(n) })
test("Groovy voluntary Java style", { n -> fib2(n) })
test("Groovy enforced Java style", { n -> fib3(n) })
j = new FibJ()
test("Java", { n -> j.fib4(n) })
$ type FibJ.java
public class FibJ {
     public int fib4(int n) {
         return n < 2 ? n : fib4(n - 1) + fib4(n - 2);
     }
}
$ javac FibJ.java
$ groovy Fib.groovy
Groovy Python style:
fib(45) = 1134903170 took 55.379 seconds.
fib(45) = 1134903170 took 50.65 seconds.
fib(45) = 1134903170 took 50.549 seconds.
Groovy voluntary Java style:
fib(45) = 1134903170 took 41.21 seconds.
fib(45) = 1134903170 took 38.139 seconds.
fib(45) = 1134903170 took 38.24 seconds.
Groovy enforced Java style:
fib(45) = 1134903170 took 3.49 seconds.
fib(45) = 1134903170 took 3.53 seconds.
fib(45) = 1134903170 took 3.38 seconds.
Java:
fib(45) = 1134903170 took 3.75 seconds.
fib(45) = 1134903170 took 3.44 seconds.
fib(45) = 1134903170 took 3.37 seconds.