classpath
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Benchmarks (who has the fastest free VM)


From: Mark Wielaard
Subject: Benchmarks (who has the fastest free VM)
Date: 08 Jul 2003 00:39:25 +0200

Hi all,

Before going to the Libre Software Meeting and LinuxTag
<http://www.libresoftwaremeeting.org/>
<http://www.linuxtag.org/2003/en/index.html>
I wanted to know how the different free VMs were doing.
So I did some micro benchmarks against them. I didn't have time to do
more complete benchmarks and I couldn't get Orp and JRVM working (which
is a shame since those are claimed to be the fasted VMs there are). So
maybe someone with more time (or access to working Orp/JRVM VMs) can do
some bigger tests.

Orp didn't compile since it has some hardcoded values for glibc 2.1
and glibc 2.2, but I am using glibc 2.3. And it needs several fixes to
get it to compile with gcc 3.x. JRVM did compile, but needs a specific
version of a proprietary VM to create the boot image. The resulting
JikesRVM binary and image keep seg faulting however on my machine.

The machine that ran the test had a Athlon 1600+ (1.4Ghz) and 768MB.
It was lightly used and I don't claim to have done scientific
benchmarking.  All times were measured with /usr/bin/time -p and the
best of three runs is given. (Note that some tests measure elapsed
time themselves so the time -p values are only for reference.)  The
benchmarks were mentioned earlier on shudo.net. If someone has more
suggestions please let me know.

VMs used:
kissme 0.0.30 (Jun 24 2003)
SableVM version 1.0.8
kaffe 1.1.x-cvs (checkout today)
ikvm (Latest ikvmbin snapshot plus Mono JIT compiler version 0.25)
gij/gcj 3.3.1 20030626 (Debian prerelease)
(For gcj tests were compiled from source using just -O2)

Looking at the results (see below) there are clearly three groups: the
plain interpreters (kissme, gij and sablevm), the jitters (kaffe and
ikvm) and the ahead of time compiler (gcj). The jitters are clearly
much faster then the interpreters, and the ahead of time compiler is
clearly much faster then the jitters.

Interesting is that ikvm has results comparable with kaffe, but often
takes more time "preparing" the classes. This can be clearly seen with
the Linpack benchmark where the ikvm score and kaffe score are almost
identical, but when looking at the real time spend you see that ikvm
consumes much more processor cycles.

SableVM recently got a "portable" jit compiler, it would be
interesting to see how it matches up to kaffe and ikvm.

Since gcj compiled code seems to be very fast but the gij interpreter
is not very fast it might be interesting to combine gcj with one of
the jitters or make one of the jitters be able to use precompiled
binaries produced by gcj (for example to precompile all core classes
used by the VM).

Let me know if you have other benchmark/speed test stories.

Cheers,

Mark


The results:

http://math.nist.gov/scimark2/

Kissme:
  Composite Score: 3.0156149099413163E174
  FFT (1024): 1.5078077289324666E175
  SOR (100x100):   -1.1107352312578372E61
  Monte Carlo : 1.6082285944663933E129
  Sparse matmult (N=1000, nz=5000): 2.6108808678857117E25
  LU (100x100): 3.5030103117925084
 
  real 26.99
  user 26.82
  sys 0.05

gij:
  Composite Score: 5.145989276632694
  FFT (1024): 2.648052903699278
  SOR (100x100):   8.881613884448855
  Monte Carlo : 1.387464107626074
  Sparse matmult (N=1000, nz=5000): 5.551639995055836
  LU (100x100): 7.261175492333425
 
  real 31.30
  user 31.19
  sys 0.10

SableVM:
  Composite Score: 7.329552111085732
  FFT (1024): 4.404292204203279
  SOR (100x100):   13.062330882926906
  Monte Carlo : 2.369663222573336
  Sparse matmult (N=1000, nz=5000): 7.329992764243664
  LU (100x100): 9.481481481481481
 
  real 28.23
  user 26.95
  sys 0.11

Kaffe:
  Composite Score: 60.46979156068268
  FFT (1024): 30.891199299334463
  SOR (100x100):   148.2273083076923
  Monte Carlo : 12.628691263262526
  Sparse matmult (N=1000, nz=5000): 54.10832237609153
  LU (100x100): 56.49343655703252
 
  real 31.17
  user 31.10
  sys 0.00

ikvm:
  Composite Score: 100.9979186697824
  FFT (1024): 59.79980402491375
  SOR (100x100):   224.2731618120265
  Monte Carlo : 34.02223834505183
  Sparse matmult (N=1000, nz=5000): 83.7413726183124
  LU (100x100): 103.1530165486078
 
  real 35.60
  user 35.42
  sys 0.09

gcj:
  Composite Score: 217.51089337171615
  FFT (1024): 225.23768378150965
  SOR (100x100):   305.57485467536026
  Monte Carlo : 15.335663953302584
  Sparse matmult (N=1000, nz=5000): 235.82583613959164
  LU (100x100): 305.5804283088167
 
  real 28.49
  user 28.44
  sys 0.06


Sieve.java from TYA <http://sax.sax.de/~adlibit/>

Kissme:
  634 iterations in 10.008 seconds
  Sieve score = 63
  real 10.10
  user 9.98
  sys 0.02

SableVM:
  2355 iterations in 10.0 seconds
  Sieve score = 236
  real 10.08
  user 10.08
  sys 0.01

gij:
  2692 iterations in 10.0 seconds
  Sieve score = 269
  real 10.10
  user 10.05
  sys 0.05

ikvm:
  40937 iterations in 10.0 seconds
  Sieve score = 4094
  real 11.32
  user 11.24
  sys 0.05

Kaffe:
  48611 iterations in 10.0 seconds
  Sieve score = 4861
  real 10.11
  user 10.10
  sys 0.01

gcj:
  91621 iterations in 10.0 seconds
  Sieve score = 9162
  real 10.03
  user 9.95
  sys 0.06


http://www.netlib.org/benchmark/linpackjava/

Kissme:
  Mflops/s: 2.289 Time: 36.62s
  Norm Res: 0.0 Precision: 2.220446049250313E-16
  real 40.73
  user 40.62
  sys 0.10

gij:
  Mflops/s: 6.078 Time: 13.79s
  Norm Res: 5.81 Precision: 2.220446049250313E-16
  real 14.74
  user 14.59
  sys 0.14

SableVM:
  Mflops/s: 8.45 Time: 9.92s
  Norm Res: 5.81 Precision: 2.220446049250313E-16
  real 10.88
  user 10.86
  sys 0.02

ikvm:
  Mflops/s: 60.312 Time: 1.39s
  Norm Res: 5.68 Precision: 2.220446049250313E-16.0
  real 5.03
  user 4.92
  sys 0.06

Kaffe:
  Mflops/s: 61.282 Time: 1.37s
  Norm Res: 4.28 Precision: 2.220446049250313E-16
  real 1.92
  user 1.89
  sys 0.03

gcj:
  Mflops/s: 79.163 Time: 1.06s
  Norm Res: 4.28 Precision: 2.220446049250313E-16
  real 1.41
  user 1.37
  sys 0.03






reply via email to

[Prev in Thread] Current Thread [Next in Thread]