[gnugo-devel] some more tests

[max-d]

I made my UI capable to run auto matches between GTP and non GTP engines
Here are some results  i got running GnuGo vs Go++ and WinHonte.

1st test
--------

 gnugo3321 level 10
go++ level 5
during this test , time was not recorded but go++ needed a lot more .


= gnugo3321cvs =10
= GO++ =20

= gnugo3321cvs won:
2 (6)
3 (6)
4 (16)
6 (3)
16 (2)
19 (13)
22 (23)
24 (7)
26 (8)
28 (1)

= GO++ won:
1 (17)
5 (6)
7 (26)
8 (45)
9 (19)
10 (31)
11 (0)
12 (25)
13 (26)
14 (54)
15 (8)
17 (1)
18 (1)
20 (3)
21 (31)
23 (12)
25 (8)
27 (46)
29 (25)
30 (6)


2nd test
---------

65 games 3.3.21 (sometimes level 12 sometimes lv 13) vs go++ level 5


= gnugo3321cvs =18
GO++ lv5 =  47

= gnugo3321cvs won:
1 (4)
10 (17)
13 (13)
15 (26)
16 (10)
20 (10)
21 (31)
23 (5)
26 (6)
30 (15)
33 (10)
51 (6)
52 (33)
54 (17)
56 (3)
58 (2)
64 (6)
65 (3)

go++  won:
2 (2)
3 (14)
4 (24)
5 (14)
6 (31)
7 (34)
8 (33)
9 (25)
11 (33)
12 (42)
14 (32)
17 (29)
18 (34)
19 (0)
22 (41)
24 (20)
25 (22)
27 (21)
28 (14)
29 (2)
31 (18)
32 (12)
34 (9)
35 (4)
36 (20)
37 (23)
38 (10)
39 (14)
40 (10)
41 (31)
42 (37)
43 (13)
44 (23)
45 (46)
46 (7)
47 (31)
48 (9)
49 (18)
50 (12)
53 (28)
55 (10)
57 (57)
59 (36)
60 (0)
61 (11)
62 (19)
63 (8)

it happens that go++ level 5 takes a lot of  time during the yose part of
every game .
Often it had a large advance at the score at the end of middle game (times
being almost equal) , but despite that it still took a lot of time to finish
the game.(see below a game which can explain why go++ needs time in end
game)

in total go++ level 5 is often very slow in comparison with gnugo 3.3.21 lv
13.

3rd test (with a slower computer)
----------------------------------------
= gnugo3_3_16 level 10 = 3
= GO++ level 5 =18

= gnugo3_3_16 won:
12 (37)
15 (14)
20 (9)

= GO++ won:
1 (6)
2 (19)
3 (26)
4 (16)
5 (18)
6 (7)
7 (18)
8 (12)
9 (4)
10 (39)
11 (13)
13 (10)
14 (34)
16 (10)
17 (18)
18 (32)
19 (6)
21 (40)

time = gnugo3_3_16: 20469
time = GO++: 39973

4th test (3 in 1)
--------

All the gnugo playing level 10 , go ++ playing level 4

1- gnugo 3.0

= gnugo3.0= 1
= GO++ =8

= gnugo3.0 won:
3 (37)

= GO++ won:
1 (44)
2 (43)
4 (49)
5 (63)
6 (47)
7 (23)
8 (88)
9 (4)

time = gnugo3.0: 2009
time = GO++: 2644

-2 gnugo.3.2

= gnugo.3.2 =3
= GO++= 6

= gnugo.3.2 won:
2 (8)
3 (22)
9 (57)

= GO++ won:
1 (23)
4 (46)
5 (6)
6 (23)
7 (72)
8 (11)

time = gnugo.3.2: 2758
time = GO++: 2947

-3 gnugo.3.3.21
= gnugo3321cvs =12
= GO++ =13

= gnugo3321cvs won:
1 (7)
2 (27)
5 (26)
6 (28)
9 (10)
10 (3)
14 (1)
15 (34)
16 (76)
17 (154)
19 (30)
22 (2)

= GO++ won:
3 (4)
4 (24)
7 (8)
8 (23)
11 (16)
12 (29)
13 (97)
18 (17)
20 (13)
21 (24)
23 (6)
24 (47)
25 (46)

time = gnugo3321cvs: 8431
time = GO++: 7341

i think that it is the more fair time settings and i will run it again for
more games .



---------------

size 13

= gnugo3321cvs 3
= GO++ 2

= gnugo3321cvs won:
1 (16)
2 (11)
5 (1)

= GO++ won:
3 (27)
4 (14)

time = gnugo3321cvs: 312
time = GO++: 428

an example of huge blunder by go++ level 4 vs gnugo 3.2

Game 9.sgf
 --mode gtp --level 10 --boardsize 19 -M 8 --komi 5.5
Size 19
Black = gnugo.3.2 White = GO++
Level black = 10 Level white = 4
Time black = 272,9 Time white = 310,8
Score = = B+57.5 (upper bound: -57.5, lower: -57.5)
(
;B[pd]
;W[dq]
;B[pp]
;W[dd]
;B[fc]
;W[cf]
;B[db]
;W[cc]
;B[ic]
;W[nc]
;B[do]
;W[kc]
;B[cq]
;W[cr]
;B[cp]
;W[er]
;B[ck]
;W[pb]
;B[ch]
;W[cb]
;B[jp]
;W[br]
;B[mq]
;W[qk]
;B[qg]
;W[qn]
;B[qo]
;W[pn]
;B[kb]
;W[lb]
;B[jb]
;W[qc]
;B[qd]
;W[hk]
;B[lc]
;W[ld]
;B[ro]
;W[ne]
;B[og]
;W[ml]
;B[la]
;W[nh]
;B[mb]
;W[if]
;B[kd]
;W[ke]
;B[pi]
;W[oj]
;B[ng]
;W[mg]
;B[hp]
;W[jd]
;B[bq]
;W[ar]
;B[jm]
;W[jk]
;B[rc]
;W[rb]
;B[rd]
;W[mf]
;B[pc]
;W[fe]
;B[ob]
;W[ri]
;B[eh]
;W[bh]
;B[bg]
;W[cg]
;B[bi]
;W[dh]
;B[ei]
;W[ci]
;B[hh]
;W[bj]
;B[rh]
;W[gl]
;B[rj]
;W[dj]
;B[hi]
;W[ih]
;B[cn]
;W[bk]
;B[dl]
;W[ej]
;B[ec]
;W[aq]
;B[fo]
;W[hr]
;B[jr]
;W[mo]
;B[mp]
;W[hn]
;B[no]
;W[mn]
;B[in]
;W[iq]
;B[hm]
;W[gm]
;B[gn]
;W[jq]
;B[kq]
;W[ir]
;B[kr]
;W[ip]
;B[jo]
;W[ca]
;B[he]
;W[hf]
;B[mc]
;W[fd]
;B[gf]
;W[hd]
;B[ge]
;W[gd]
;B[hc]
;W[oi]
;B[qj]
;W[pj]
;B[qi]
;W[kl]
;B[rk]
;W[ql]
;B[nd]
;W[em]
;B[oe]
;W[nn]
;B[oo]
;W[dm]
;B[cm]
;W[go]
;B[ho]
;W[el]
;B[gp]
;W[fq]
;B[nf]
;W[bl]
;B[cl]
;W[me]
;B[ap]
;W[bo]
;B[bp]
;W[bn]
;B[bm]
;W[da]
;B[ea]
;W[ed]
;B[ie]
;W[id]
;B[ff]
;W[jf]
;B[gc]
;W[lo]
;B[ep]
;W[am]
;B[an]
;W[al]
;B[ao]
;W[eq]
;B[rn]
;W[rl]
;B[km]
;W[lm]
;B[kn]
;W[il]
;B[en]
;W[dp]
;B[co]
;W[fm]
;B[on]
;W[om]
;B[jc]
;W[rm]
;B[fp]
;W[lp]
;B[dc]
;W[lq]
;B[lr]
;W[oh]
;B[ph]
;W[md]
;B[od]
;W[js]
;B[gq]
;W[gr]
;B[ks]
;W[is]
;B[dk]
;W[ek]
;B[im]
;W[kp]
;B[mr]
;W[sk]
;B[sj]
;W[sl]
;B[io]
;W[sn]
;B[so]
;W[hg]
;B[kd]
;W[dn]
;B[kc]
;W[fn]
;B[eo]
;W[sm]
;B[jl]
;W[ik]
;B[kk]
;W[ll]
;B[je]
;W[cj]
;B[po]
;W[ln]
;B[pm]
;W[qm]
;B[ol]
;W[ok]
;B[nm]
;W[nl]
;B[pk]
;W[hl]
;B[pl]
;B[ko]
;W[hq]
;B[om]
;B[mm]
;W[kj]
;B[kf]
;W[kg]
;B[le]
;W[gg]
;B[fg]
;W[lf]
;B[ke]
;W[gh]
;B[mh]
;W[mi]
;B[lg]
;W[kh]
;B[fi]
;W[gi]
;B[lh]
;W[li]
;B[ef]
;W[fj]
;B[di]
;W[de]
;B[dg]
;W[ch]
;B[df]
;W[fh]
;B[eg]
;W[ee]
)
----------------------------------------------------------------------------
------
VS WinHonte 0.94
WH settings were 10-8-6

test 1
------
sz 19
= gnugo.3.2 =1
= WinHonte =10

= gnugo.3.2 lv 10 won:
1 (163)

= WinHonte won:
2 (11)
3 (20)
4 (43)
5 (5)
6 (38)
7 (24)
8 (20)
9 (38)
10 (19)
11 (10)

time = gnugo.3.2: 3901
time = WinHonte: 4660

test 2
-------
sz19

= gnugo3321cvs lv 10 = 24
= WinHonte= 7

= gnugo3321cvs won:
1 (116)
2 (47)
4 (43)
6 (3)
7 (38)
8 (91)
9 (40)
10 (58)
11 (22)
12 (11)
13 (117)error :game aborted
15 (63)
18 (29)
20 (85)
21 (14)
22 (1)
23 (11)
24 (179)
25 (18)
26 (7)
27 (54)
28 (41)
29 (39)
30 (93)

= WinHonte won:
3 (32)
5 (88)
14 (25)
16 (5)
17 (18)
19 (2)
31 (7)

time = gnugo3321cvs: 12960
time = WinHonte: 12136

test 3
-------
size 13

= gnugo3321cvs 8
= WinHonte 1

= gnugo3321cvs won:
1 (32)
3 (6)
4 (11)
5 (37)
6 (0)
7 (11)
8 (23)
9 (60)

= WinHonte won:
2 (12)

time = gnugo3321cvs: 826
time = WinHonte: 1023

test 4
sz 9

= gnugo3321cvs 11
= WinHonte 2

= gnugo3321cvs won:
1 (21)
3 (7)
4 (7)
5 (3)
6 (20)
7 (2)
8 (2)
9 (2)
11 (11)
12 (3)
13 (9)

= WinHonte won:
2 (20)
10 (4)

time = gnugo3321cvs: 419
time = WinHonte: 598