Re: elisp-benchmarks

[Stefan Monnier]

> Thank you! Normalised to the first version where each benchmark result
> appears, we have (excluding GC):
>
> |                    |   26 |   27 |   28 | pre sympos | sympos | master |
> |--------------------+------+------+------+------------+--------+--------|
> | bubble             | 1.00 | 0.81 | 0.54 |       0.62 |   0.58 |   0.46 |
> | bubble-no-cons     | 1.00 | 0.87 | 0.75 |       0.77 |   0.78 |   0.63 |
> | bytecomp           | 1.00 | 0.94 | 1.00 |       1.07 |   1.13 |   1.06 |
> | dhrystone          | 1.00 | 0.91 | 0.82 |       0.83 |   0.86 |   0.66 |
> | eieio              | 1.00 | 0.98 | 0.95 |       1.04 |   1.03 |   0.87 |
> | fibn               | 1.00 | 0.86 | 0.75 |       0.75 |   0.75 |   0.52 |
> | fibn-named-let     |      |      | 1.00 |       1.03 |   1.01 |   0.70 |
> | fibn-rec           | 1.00 | 1.14 | 1.00 |       1.02 |   1.04 |   0.68 |
> | fibn-tc            | 1.00 | 1.05 | 0.96 |       0.99 |   1.00 |   0.59 |
> | flet               | 1.00 | 1.23 | 0.88 |       0.91 |   0.96 |   0.62 |
> | inclist            | 1.00 | 1.36 | 0.79 |       0.78 |   0.84 |   0.76 |
> | inclist-type-hints | 1.00 | 1.40 | 0.60 |       0.59 |   0.64 |   0.58 |
> | listlen-tc         | 1.00 | 0.97 | 0.89 |       0.91 |   0.92 |   0.67 |
> | map-closure        | 1.00 | 1.14 | 1.04 |       1.04 |   1.06 |   0.72 |
> | nbody              | 1.00 | 0.95 | 0.82 |       0.84 |   0.81 |   0.71 |
> | pack-unpack        |      |      | 1.00 |       1.04 |   1.03 |   0.83 |
> | pack-unpack-old    | 1.00 | 0.89 | 0.49 |       0.52 |   0.52 |   0.47 |
> | pcase              | 1.00 | 0.86 | 0.72 |       0.74 |   0.64 |   0.63 |
> | pidigits           |      | 1.00 | 1.06 |       1.15 |   1.07 |   0.99 |
> | scroll             |      |      | 1.00 |       1.05 |   1.02 |   1.09 |
> | smie               | 1.00 | 0.86 | 0.78 |       0.77 |   0.79 |   0.78 |

Thanks.

> (The pidigits result was omitted from Emacs 26 because it didn't make much 
> sense.)

Good catch, yes, `pidigits` needs bignums to make sense, otherwise it
computes (quickly) a bogus result.

> The bytecomp benchmark is mobile -- it does more work (and generates better
> code) now than in earlier versions.

Yes, we don't want `bytecomp` to become unbearably so, it's OK if it
gets a bit slower if we get better code or better error messages out of it.

> Not sure how comparable scroll is, but it's probably a difficult thing to 
> measure.

I don't know how to interpret the evolution of that benchmark, indeed.
It paradoxically seems to be made faster by sympos and slower by your
subsequent bytecode improvements.  I'd tend to assume that the "faster
with sympos" is true-or-negligible, and but the "slower on master" is
a bit more perplexing.  Could it be that there have been other changes that
happened on master recently that would cause that?
My intuition is that neither sympos nor your bytecode work should make
very much difference to the `scroll` benchmark since it should spend
most of its time in other parts of the C code (e.g. text-property
manipulations, regexp matching, redisplay, string extraction from the
buffer, ...).


        Stefan