[gnugo-devel] breakin code discussion

[Arend Bayer]

While I know that the break-in code is a tradeoff, and I understand either
position about it iwth regards to 3.4, I have to admit that I am irritated
by the discussion in the last days. It ignored all the data points I had
given in earlier mails to enable an informed discussion about it.

I had spent couple of hours into these, and if my mails don't get read
I don' quite know why I am writing these e-mails to explain what I am doing.

1. "The break-in code costs 25% performance in games."

I think the best effort to measure the performance cost is still
http://mail.gnu.org/archive/html/gnugo-devel/2003-06/msg00198.html.
(Which btw also shows that we are still a lot faster than 3.2).
(I don't think anything with major performance impact has happened 
since then.)

This is certainly not enough for a reliable benchmark, since too few
games got replayed, but it is definitely better than just taking timings
from gnugo-vs-gnugo matches, on which the above opinion seems to be
based.

I don't want to be impolite, but I think anyone interested in gnugo's
performance could have noticed that gnugo-vs-gnugo games are typically
very fast, a lot faster than NNGS games (which are btw still _a lot_
faster than when I play a high handicap game against gnugo myself -- these
are probably worst case performance wise). The reason for this is that
twogtp matches are far more peaceful than typical games, and hence owl
costs are far lower than usual. Thus the costs of the connection code and
the break-in are overvalued there.
(For the same reason I feel that the recent performance debate on the list 
is pretty much lacking facts to be based on.)

2.  Evaluating the benefit of the break-in code

a) It doesn't help in gnugo vs gnugo matches. 
While this is true, the numbers recently cited for this are obviously based
on too few games. Thanks to Inge I can refer to
http://mail.gnu.org/archive/html/gnugo-devel/2003-07/msg00027.html
instead.

b) NNGS ratings.
They climbed by appr. 0.25 after 3.3.21 and once again somewhat later (after
an important break-in fix got in, I believe this was arend_3_21.1 but I
cannot check as the link is broken).
I don't give too much on this since NNGS ratings sometimes vary quite a bit.

c) Regressions
I haven't run these for a while, but these were fairly positive for
the break-in code.

d) By-hand evaluation
http://mail.gnu.org/archive/html/gnugo-devel/2003-07/msg00067.html
(and follow-ups).
This is still the one I trust the most. The only problem with this is that
it's of course subjective, and someone else cannot check my conclusions
without spending a similar amount of time.

Why doesn't it help in gnugo-vs-gnugo matches? I think it is a similar
reason as why they are fast. They tend to be somewhat no-contact games,
and hence the break-in code simply doesn't make a difference.
It is of course disappointing that it still consumes 20% of run time
in this case.

Arend