Re: [gnugo-devel] 261 games 3.1.31 vs 3.1.32

[Arend Bayer]

On Fri, 19 Apr 2002, Marco Scheurer wrote:

> 
> On Thursday, April 18, 2002, at 08:17  pm, <address@hidden> 
> wrote:
> 
> > Since shortly after the 3.1.31 release, we've had a precipitous
> > rating slide on NNGS. I did some testing to see if 3.1.31 showed
> > signs of being broke.
Of course an easy explanation for this rating slide is the renewed interest
of Lazarus in GNU Go... (That is not to say we should not be careful with
this.)

> And in general, is playing against older versions of GNU Go a good test? 
> While testing Goban, I've been looking at a few games between different 
> versions of GNU Go, and they seemed strange to me (but I'm a weak 
> player). It would be nice to have other comparisons, like the game 
> series against Handtalk. I don't have easy access to a Windows PC 
> outside of work, but I'm thinking about investing in SoftPC for my Mac 
> just to match GNU Go against Smart Go or Many Faces or Go Professional.

...whereas I always feel like buying a Mac when I see the screen shots
of Sen:te Goban :)

I fully agree that it would be more meaningful to test against other
opponent's. (E.g. I wouldn't take the observed 3-stones difference
between current GNU Go and 3.0.0 for granted.) However, say if we would
assume a loss in strength of half a stone, then we would need quite a lot
of games to detect this reliably via a game series.
As I don't think the commercial versions of the top programs include GMP
(not to speak of GTP), you cannot do such a series automatically.

On the other hand, every game against another computer opponent will be
useful for us. I have already started creating test suite from the Handtalk
game series, and the games promise to be very good for tuning.
(Thanks a lot Jessie!!)
I am sure we could similarly learn a lot from games against Go Professional
et. al.

Arend