Re: [lmi] Benchmarking: gcc-8 beats gcc-10 soundly?

[Greg Chicares]

On 2020-09-19 22:04, Vadim Zeitlin wrote:
[...]
> 1. Check if we can see the regression under Linux too because:
>  (a) I think gcc developers would pay more attention to something which
>      is not MSW/MinGW-specific.

Wait...I thought the regression might be partly in 'wine'
and partly in MinGW-w64-gcc, but it's actually all in 'wine'.
Here are timings for all six '--selftest' scenarios, all run
in the same chroot with wine-4.0:

gcc-8  gcc-10   ratio
51127   51476   1.007
91813   92247   1.005
84345   84630   1.003
20626   20742   1.006
58064   58173   1.002
53847   54107   1.005

Thus, we can confidently upgrade to gcc-10: it passes our
full nychthemeral test suite, and it's a negligible half
of a percent slower.

Here's the wine-5 vs wine-4 regression. The first column
is copied from above. The second uses the same gcc-10-built
binary as above, but is run in a different chroot that has
wine-5 instead of wine-4:

formula* gcc-8  gcc-10 ratio difference
51781    51127   66924  1.31    15797
91618    91813  109947  1.20    18134
84418    84345  102171  1.21    17826
20604    20626   33252  1.61    12626
57517    58064   73118  1.26    15054
53284    53847   68547  1.27    14700

The ratios vary considerably; the differences, less so.
It looks like the wine regression adds a fixed cost of
about 11000 us; subtracting that, it's still about eight
percent slower. Thus...

* The "formula" column is
  (gcc10 - 11000) / 1.08
which is a pretty good fit to the gcc-8 data.