[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
elisp-benchmarks
|
From: |
Stefan Monnier |
|
Subject: |
elisp-benchmarks |
|
Date: |
Wed, 09 Feb 2022 17:19:28 -0500 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) |
Hi,
I just ran the elisp-benchmarks (with a repeat count of 10) on some
little-used server we have here and figured others might like to see the
result:
Before sympos (i.e. commit f57f28935a1f0c46776976dd497322a07d248f5f)
| test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) |
tot avg err (s) |
|--------------------+----------------+------------+---------+-------------+-----------------|
| bubble | 6.75 | 8.75 | 433 | 15.50 |
0.03 |
| bubble-no-cons | 16.26 | 0.06 | 3 | 16.32 |
0.00 |
| bytecomp | 3.24 | 2.53 | 123 | 5.77 |
0.16 |
| dhrystone | 15.54 | 0.00 | 0 | 15.54 |
0.00 |
| eieio | 5.57 | 3.38 | 166 | 8.95 |
0.02 |
| fibn | 5.45 | 0.00 | 0 | 5.45 |
0.00 |
| fibn-named-let | 6.00 | 0.00 | 0 | 6.00 |
0.01 |
| fibn-rec | 11.11 | 0.00 | 0 | 11.11 |
0.01 |
| fibn-tc | 9.24 | 0.00 | 0 | 9.24 |
0.00 |
| flet | 18.50 | 0.00 | 0 | 18.50 |
0.01 |
| inclist | 18.11 | 0.00 | 0 | 18.11 |
0.01 |
| inclist-type-hints | 18.11 | 0.00 | 0 | 18.11 |
0.01 |
| listlen-tc | 9.33 | 0.00 | 0 | 9.33 |
0.02 |
| map-closure | 14.49 | 0.00 | 0 | 14.49 |
0.01 |
| nbody | 5.98 | 16.34 | 791 | 22.32 |
0.06 |
| pack-unpack | 1.07 | 0.87 | 43 | 1.94 |
0.00 |
| pack-unpack-old | 3.29 | 1.76 | 87 | 5.06 |
0.01 |
| pcase | 20.28 | 0.00 | 0 | 20.28 |
0.00 |
| pidigits | 11.96 | 10.17 | 401 | 22.13 |
0.18 |
| scroll | 1.34 | 0.12 | 6 | 1.46 |
0.00 |
| smie | 3.06 | 0.95 | 46 | 4.00 |
0.01 |
|--------------------+----------------+------------+---------+-------------+-----------------|
| total | 204.67 | 44.94 | 2102 | 249.61 |
0.26 |
Right after sympos (i.e. commit df49e3a3ab4cddf1e3c0f5482c7fdd809d8a8884)
| test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) |
tot avg err (s) |
|--------------------+----------------+------------+---------+-------------+-----------------|
| bubble | 6.39 | 9.09 | 449 | 15.48 |
0.04 |
| bubble-no-cons | 16.50 | 0.06 | 3 | 16.56 |
0.00 |
| bytecomp | 3.43 | 2.69 | 130 | 6.12 |
0.16 |
| dhrystone | 16.07 | 0.00 | 0 | 16.07 |
0.01 |
| eieio | 5.50 | 3.38 | 166 | 8.88 |
0.02 |
| fibn | 5.42 | 0.00 | 0 | 5.42 |
0.00 |
| fibn-named-let | 5.87 | 0.00 | 0 | 5.87 |
0.00 |
| fibn-rec | 11.27 | 0.00 | 0 | 11.27 |
0.01 |
| fibn-tc | 9.42 | 0.00 | 0 | 9.42 |
0.00 |
| flet | 19.65 | 0.00 | 0 | 19.65 |
0.01 |
| inclist | 19.43 | 0.00 | 0 | 19.43 |
0.00 |
| inclist-type-hints | 19.43 | 0.00 | 0 | 19.43 |
0.00 |
| listlen-tc | 9.35 | 0.00 | 0 | 9.35 |
0.01 |
| map-closure | 14.83 | 0.00 | 0 | 14.83 |
0.05 |
| nbody | 5.78 | 16.38 | 789 | 22.15 |
0.07 |
| pack-unpack | 1.06 | 0.87 | 43 | 1.93 |
0.00 |
| pack-unpack-old | 3.32 | 1.77 | 86 | 5.09 |
0.01 |
| pcase | 17.65 | 0.00 | 0 | 17.65 |
0.00 |
| pidigits | 11.15 | 10.06 | 400 | 21.21 |
0.18 |
| scroll | 1.30 | 0.12 | 6 | 1.42 |
0.00 |
| smie | 3.11 | 0.95 | 46 | 4.06 |
0.00 |
|--------------------+----------------+------------+---------+-------------+-----------------|
| total | 205.92 | 45.37 | 2122 | 251.29 |
0.26 |
>From master (i.e. commit d41a5e7e33067eb38b147ee2f8a1615f6faed7a4)
| test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) |
tot avg err (s) |
|--------------------+----------------+------------+---------+-------------+-----------------|
| bubble | 5.00 | 9.15 | 433 | 14.15 |
0.07 |
| bubble-no-cons | 13.43 | 0.06 | 3 | 13.49 |
0.00 |
| bytecomp | 3.22 | 2.65 | 123 | 5.87 |
0.16 |
| dhrystone | 12.37 | 0.00 | 0 | 12.37 |
0.03 |
| eieio | 4.65 | 3.40 | 160 | 8.05 |
0.06 |
| fibn | 3.74 | 0.00 | 0 | 3.74 |
0.00 |
| fibn-named-let | 4.10 | 0.00 | 0 | 4.10 |
0.00 |
| fibn-rec | 7.33 | 0.00 | 0 | 7.33 |
0.00 |
| fibn-tc | 5.58 | 0.00 | 0 | 5.58 |
0.00 |
| flet | 12.55 | 0.00 | 0 | 12.55 |
0.01 |
| inclist | 17.67 | 0.00 | 0 | 17.67 |
0.01 |
| inclist-type-hints | 17.67 | 0.00 | 0 | 17.67 |
0.01 |
| listlen-tc | 6.84 | 0.00 | 0 | 6.84 |
0.01 |
| map-closure | 10.06 | 0.00 | 0 | 10.06 |
0.00 |
| nbody | 5.08 | 16.51 | 762 | 21.59 |
0.04 |
| pack-unpack | 0.86 | 0.88 | 41 | 1.74 |
0.01 |
| pack-unpack-old | 2.99 | 1.78 | 83 | 4.77 |
0.02 |
| pcase | 17.36 | 0.00 | 0 | 17.36 |
0.00 |
| pidigits | 10.32 | 9.97 | 386 | 20.28 |
0.34 |
| scroll | 1.39 | 0.13 | 6 | 1.52 |
0.00 |
| smie | 3.10 | 0.95 | 44 | 4.06 |
0.02 |
|--------------------+----------------+------------+---------+-------------+-----------------|
| total | 165.30 | 45.49 | 2044 | 210.78 |
0.39 |
`bytecomp`, `scroll`, and `smie` are benchmarks I added yesterday which
are not microbenchmarks and thus hopefully reflect "real use".
This was all with the byte-compiler and not the native compiler.
These result suggest that the symbol-with-pos changes don't introduce
a significant slowdown, even for the bytecompiler.
And we see that Matthias's recent improvements to the bytecode
interpreter do make a quite significant difference on several of those
microbenchmarks ;-), and also on the bytecompiler benchmark (offsetting
the extra work needed for the symbol-with-positions) tho they don't make
much of a difference when it comes to scrolling(with-jit-lock) or when
it comes to reindenting code with SMIE :-)
The >10% slowdown recently seen on the test suite is still a mystery
waiting for someone to figure out what's going on.
BTW, I think one thing is clear when I look at those benchmarks:
Emacs's GC is not good enough.
Stefan
- elisp-benchmarks,
Stefan Monnier <=