RE: [gnugo-devel] More on minimum values

[Arend Bayer]

Nando wrote:
> Taking sd's into account, my conclusions concerning my patch are
> following :
> - in the average, it has possibly improved the playing strength
>   by about 2~3 more games won out of 100.
> - the most optimistic view could be an improvement of 7
> - the worst case is a possible degradation of 1~2
>
> Since the worst case is still negative, I'm rejecting my patch in its
> current form and trying something else.

I think you are setting the bar too high here. We can't do a
scientifically founded 95%-confidence statistical evaluation of each
patch we are considering. There are simply too many small changes that
it takes to make GNU Go better. So we always have to do some educated
guessing.

Also, don't forget a-priori-odds. E.g. if you take 5 random patches
which are just experiments, then take the one with the best twogtp
result, then I'd indeed be skeptical of such a result. However, if you
have good reasons beforehand to believe your patch does s.th. good, then
I'd take the results you quote as pretty encouraging.

But yes, 500 games are of course better than 100.

Arend