[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[DotGNU]Speed comparison ground rules
From: |
Rhys Weatherley |
Subject: |
[DotGNU]Speed comparison ground rules |
Date: |
Thu, 21 Nov 2002 14:35:24 +1000 |
j_post wrote:
> My app compiled with cscc and executed with ilrun runs in 10 seconds. Same
> source compiled with mcs and executed with mono runs in 36 seconds.
OK, some ground rules on speed comparisons, so we don't end up
with a "my engine is faster than your engine" slanging match
with Mono before the week is out ( :-) ):
- Always check the output of the compilers with ildasm to see if
there is some subtle difference that is contributing to the
speed difference. e.g. the "ldlen" thing with "qfractals".
- Try to factor out load-time overhead. i.e. "time ilrun app.exe"
may appear to be faster only because our C# library is currently
smaller and so loads quicker. Kernel cache mechanisms can also
play a role during loading. Measure times after load using
"DateTime.Now". Load-time overhead is important, but not on
benchmark comparisons.
- Report exactly what engine and platform you are using. e.g.
"mono" vs "mint"; "unrolled ilrun" vs "switch-loop ilrun".
- Report whether the app is I/O bound, CPU bound, network bound,
or internalcall bound.
- If you don't supply the source code for the test, we won't
believe the numbers.
- Realise that discrepancies will occur because there is no such
thing as the "fastest possible bytecode engine". The problem
domain always plays a role.
- Remember that 10 seconds after you post "A is faster than B",
the developers on B will change the code to make the statement
false. :-)
Cheers,
Rhys.