Re: [gnugo-devel] some more tests

[max-d]

----- Original Message -----
From: "Gunnar Farneback" <address@hidden>
To: "GNU Go development" <address@hidden>
Sent: Monday, July 28, 2003 9:01 PM
Subject: Re: [gnugo-devel] some more tests


> Max wrote:
> > the problem is go++ loses a lot of time at the end of the game so i
> > think the more fair way to compare both programs is to use truncated
> > games letting them play say 200 moves in total ,gnugo playing level
> > 12-13 and go++ level 5. At this stage ,the time will be almost equal.
> > Then let Gnugo estimates the score . I would say this one will be
> > often in favor of Go++
>
> Are you saying that Go++ is playing both worse and substantially
> slower than GNU Go in the endgame?

No !
in the endgame , Go++ does waste a lot of time for computing hopeless moves
and gnugo responds very quickly .
So in most of games against Gnugo level 13 , go++ level 5 is far ahead in
time
after say 200 moves and is far behind at the end of the game.
So Gnugo level 13 seems to be faster than go++ level 5 but it is an illusion
,
 the more important part of the game having been played faster by go++ lv5.

It is difficult to find the more fair settings concerning time for the 2
softwares
i think level 10 vs go++ level 4 is the good choice (at least for
benchmarking for further GnuGo versions ).


That's certainly encouraging and
> shows that the work put into the endgame by a number of people has
> paid off.

I have not watch to many games but i have seen some go++'s  blunders in
endgame
i reported one few days ago .
I suppose Gnugo has an analyse feature which shows the estimated score
evotution all
along the game.


>
> > which is probably the stronger strategical program (at the current
> > stage of gnugo devellopment ) .
>
> No question about that. Strategical understanding is known to be a
> weak spot of GNU Go currently.
>
> Thanks for doing these tests. It would be nice if you could also post
> a couple of sample games.

most of the games are here
http://perso.wanadoo.fr/gib.explore/go/games2.zip

in the case you can't open zip files , here are 5 games for which endgame
had huge value


Game 19.sgf
 --mode gtp --level 12 --boardsize 19 -M 8 --komi 5.5
Size 19
Black = gnugo3321cvs White =
Level black = 12 Level white = 0
Time black = 0 Time white = 706,9
Score = = W+0.5 (upper bound: 0.5, lower: 0.5)
(
;B[qd]
;W[dq]
;B[co]
;W[dd]
;B[ep]
;W[eq]
;B[fp]
;W[gq]
;B[cj]
;W[qq]
;B[ho]
;W[nc]
;B[pc]
;W[kc]
;B[df]
;W[fc]
;B[cd]
;W[cc]
;B[ce]
;W[bc]
;B[iq]
;W[cp]
;B[do]
;W[qj]
;B[pp]
;W[pq]
;B[op]
;W[nr]
;B[ro]
;W[qg]
;B[ff]
;W[hd]
;B[hg]
;W[rp]
;B[lr]
;W[qm]
;B[om]
;W[bp]
;B[bo]
;W[gr]
;B[qp]
;W[rq]
;B[bd]
;W[re]
;B[pf]
;W[rd]
;B[rc]
;W[ir]
;B[lp]
;W[jq]
;B[ip]
;W[jp]
;B[jo]
;W[ao]
;B[an]
;W[ap]
;B[bm]
;W[rn]
;B[nk]
;W[lk]
;B[qe]
;W[qf]
;B[qo]
;W[so]
;B[nq]
;W[oq]
;B[ik]
;W[lh]
;B[ii]
;W[lf]
;B[mr]
;W[ob]
;B[km]
;W[mk]
;B[pk]
;W[qk]
;B[pb]
;W[mm]
;B[oc]
;W[nb]
;B[nf]
;W[if]
;B[or]
;W[pr]
;B[ns]
;W[kr]
;B[kp]
;W[ad]
;B[ae]
;W[ac]
;B[oi]
;W[ph]
;B[oa]
;W[na]
;B[pa]
;W[ne]
;B[oe]
;W[nd]
;B[de]
;W[ed]
;B[sd]
;W[rf]
;B[qn]
;W[rm]
;B[oh]
;W[nn]
;B[kq]
;W[jr]
;B[gp]
;W[ig]
;B[ih]
;W[jh]
;B[hq]
;W[hr]
;B[pg]
;W[pi]
;B[pl]
;W[kl]
;B[jl]
;W[kn]
;B[jm]
;W[lm]
;B[jn]
;W[mn]
;B[nl]
;W[ge]
;B[ji]
;W[ki]
;B[kj]
;W[lj]
;B[kk]
;W[hf]
;B[gg]
;W[mf]
;B[ng]
;W[no]
;B[ko]
;W[ee]
;B[ef]
;W[ll]
;B[ln]
;W[mp]
;B[np]
;W[mo]
;B[pj]
;W[nj]
;B[mi]
;W[li]
;B[ql]
;W[rl]
;B[se]
;W[rh]
;B[mj]
;W[mg]
;B[mh]
;W[mq]
;B[sf]
;W[sg]
;B[sc]
;W[od]
;B[fq]
;W[fr]
;B[lo]
;W[fe]
;B[ks]
;W[js]
;B[ls]
;W[ps]
;B[pd]
;W[os]
;B[nr]
;W[on]
;B[lq]
;W[af]
;B[be]
;W[gf]
;B[dp]
;W[cq]
;B[nm]
;W[ni]
;B[nh]
;W[jj]
;B[jk]
;W[pm]
;B[pn]
;W[oo]
;B[ml]
;W[ok]
;B[oj]
;B[po]
)

Game 60.sgf
 --mode gtp --level 12 --boardsize 19 -M 8 --komi 5.5
Size 19
Black = gnugo3321cvs White = GO++
Level black = 12 Level white = 0
Time black = 346,3 Time white = 462,9
Score = = W+0.5 (upper bound: 0.5, lower: 0.5)
(
;B[pd]
;W[cp]
;B[pq]
;W[dd]
;B[eq]
;W[cm]
;B[hq]
;W[qn]
;B[qp]
;W[qk]
;B[cf]
;W[fc]
;B[bd]
;W[cc]
;B[ci]
;W[qh]
;B[hc]
;W[bc]
;B[kc]
;W[dq]
;B[ep]
;W[le]
;B[md]
;W[qc]
;B[pc]
;W[qd]
;B[qe]
;W[re]
;B[qf]
;W[rf]
;B[oo]
;W[me]
;B[of]
;W[mo]
;B[ph]
;W[jn]
;B[qi]
;W[ri]
;B[rh]
;W[qj]
;B[pi]
;W[sh]
;B[kq]
;W[ki]
;B[rg]
;W[pb]
;B[ob]
;W[sg]
;B[nq]
;W[qb]
;B[qg]
;W[rj]
;B[od]
;W[jf]
;B[bk]
;W[dr]
;B[er]
;W[jp]
;B[jq]
;W[es]
;B[fs]
;W[ds]
;B[om]
;W[mm]
;B[fi]
;W[hi]
;B[gk]
;W[ik]
;B[be]
;W[ef]
;B[fr]
;W[id]
;B[ic]
;W[jd]
;B[jc]
;W[hd]
;B[dl]
;W[fh]
;B[eh]
;W[gh]
;B[eg]
;W[fg]
;B[mi]
;W[lj]
;B[gb]
;W[fb]
;B[ro]
;W[rn]
;B[pa]
;W[bl]
;B[cl]
;W[bm]
;B[al]
;W[mj]
;B[mh]
;W[df]
;B[dg]
;W[ip]
;B[iq]
;W[gc]
;B[am]
;W[an]
;B[ak]
;W[bo]
;B[hb]
;W[hm]
;B[fa]
;W[ea]
;B[ga]
;W[eb]
;B[sn]
;W[sm]
;B[so]
;W[rm]
;B[qa]
;W[ra]
;B[oa]
;W[rb]
;B[hp]
;W[hk]
;B[gl]
;W[ld]
;B[lc]
;W[lg]
;B[ne]
;W[ho]
;B[go]
;W[hn]
;B[gj]
;W[kp]
;B[lp]
;W[lo]
;B[mp]
;W[ac]
;B[ce]
;W[do]
;B[nl]
;W[ml]
;B[nj]
;W[mg]
;B[ng]
;W[dm]
;B[em]
;W[en]
;B[el]
;W[fo]
;B[fn]
;W[eo]
;B[gn]
;W[fp]
;B[no]
;W[gq]
;B[gr]
;W[qo]
;B[rp]
;W[pl]
;B[de]
;W[ee]
;B[hj]
;W[ij]
;B[nn]
;W[po]
;B[mn]
;W[ln]
;B[hl]
;W[il]
;B[gm]
;W[pp]
;B[op]
;W[pm]
;B[lh]
;W[kh]
;B[kd]
;W[ke]
;B[mk]
;W[lk]
;B[nk]
;W[mf]
;B[nf]
;W[ad]
;B[ae]
;W[ni]
;B[nh]
;W[oj]
;B[oi]
;W[nm]
;B[ol]
;W[pj]
;B[ok]
;W[gp]
;B[gi]
;W[hh]
;B[dp]
;W[qq]
;B[rq]
;W[pr]
;B[oq]
;W[qr]
;B[rr]
;W[hr]
;B[ir]
;W[fq]
;B[hs]
;W[cd]
;B[pk]
;W[ch]
;B[cg]
;W[ck]
;B[cj]
;W[bj]
;B[dk]
;W[bh]
;B[on]
;W[li]
;B[pn]
;W[rk]
;W[aj]
;B[ck]
;W[mc]
;B[nd]
;W[rs]
;B[sr]
;W[or]
;B[nr]
;W[ms]
;B[ns]
)


Game 58.sgf
 --mode gtp --level 12 --boardsize 19 -M 8 --komi 5.5
Size 19
Black = gnugo3321cvs White = GO++
Level black = 12 Level white = 0
Time black = 426,5 Time white = 358,6
Score = = B+2.5 (upper bound: -2.5, lower: -2.5)
(
;B[pd]
;W[dd]
;B[cp]
;W[qq]
;B[df]
;W[fc]
;B[cd]
;W[cc]
;B[ce]
;W[bc]
;B[cj]
;W[eq]
;B[do]
;W[hq]
;B[fp]
;W[fq]
;B[pn]
;W[nq]
;B[ho]
;W[qi]
;B[qk]
;W[qf]
;B[nc]
;W[rd]
;B[qc]
;W[rc]
;B[kc]
;W[ic]
;B[ej]
;W[qo]
;B[qn]
;W[rn]
;B[jp]
;W[cr]
;B[bd]
;W[rm]
;B[ff]
;W[ie]
;B[ok]
;W[mn]
;B[jn]
;W[bq]
;B[bp]
;W[kj]
;B[ij]
;W[oi]
;B[jl]
;W[ll]
;B[cq]
;W[br]
;B[of]
;W[mi]
;B[hg]
;W[nm]
;B[om]
;W[rb]
;B[qb]
;W[mg]
;B[gp]
;W[jq]
;B[kq]
;W[md]
;B[mc]
;W[ld]
;B[jb]
;W[ib]
;B[lp]
;W[jr]
;B[gq]
;W[gr]
;B[np]
;W[no]
;B[mq]
;W[op]
;B[mp]
;W[oq]
;B[hr]
;W[hs]
;B[fr]
;W[er]
;B[gs]
;W[es]
;B[nr]
;W[or]
;B[mr]
;W[pg]
;B[os]
;W[ps]
;B[ns]
;W[pr]
;B[ra]
;W[jg]
;B[ji]
;W[ki]
;B[ig]
;W[jh]
;B[aq]
;W[ar]
;B[ap]
;W[og]
;B[nd]
;W[nf]
;B[kd]
;W[le]
;B[ia]
;W[ha]
;B[ja]
;W[gb]
;B[oe]
;W[ad]
;B[ae]
;W[ac]
;B[de]
;W[ed]
;B[ke]
;W[kf]
;B[ih]
;W[ge]
;B[gf]
;W[jc]
;B[lb]
;W[nl]
;B[jj]
;W[kk]
;B[ne]
;W[me]
;B[ep]
;W[dq]
;B[fe]
;W[he]
;B[fd]
;W[gd]
;B[qe]
;W[re]
;B[jk]
;W[fs]
;B[gr]
;W[ee]
;B[ef]
;W[km]
;B[jm]
;W[kn]
;B[ko]
;W[qd]
;B[pe]
;W[ka]
;B[kb]
;W[pb]
;B[pc]
;W[jd]
;B[ln]
;W[lm]
;B[lo]
;W[sb]
;B[je]
;W[jf]
;B[lc]
;W[sa]
;B[qa]
;W[af]
;B[be]
;W[pf]
;B[if]
;W[mo]
;B[dp]
;W[hf]
;B[kl]
)

Game 2.sgf
 --mode gtp --level 12 --boardsize 19 -M 8 --komi 5.5
Size 19
Black = gnugo3321cvs White =
Level black = 12 Level white = 0
Time black = 0 Time white = 1117,0
Score = = W+2.5
(
;B[pd]
;W[dq]
;B[dd]
;W[pp]
;B[do]
;W[co]
;B[dp]
;W[cp]
;B[eq]
;W[cq]
;B[dn]
;W[cm]
;B[fp]
;W[jc]
;B[np]
;W[qn]
;B[pq]
;W[qq]
;B[oq]
;W[qr]
;B[jq]
;W[qf]
;B[pf]
;W[qc]
;B[pc]
;W[qd]
;B[pg]
;W[cf]
;B[el]
;W[ci]
;B[fd]
;W[mc]
;B[qg]
;W[rf]
;B[be]
;W[bf]
;B[ce]
;W[jl]
;B[pr]
;W[dk]
;B[cn]
;W[bn]
;B[rg]
;W[pb]
;B[ob]
;W[qb]
;B[nc]
;W[mb]
;B[qj]
;W[df]
;B[mf]
;W[gc]
;B[fc]
;W[gb]
;B[fb]
;W[ql]
;B[ff]
;W[ae]
;B[ad]
;W[af]
;B[bc]
;W[fa]
;B[ea]
;W[ga]
;B[lp]
;W[jo]
;B[hp]
;W[hn]
;B[gd]
;W[hd]
;B[oj]
;W[gl]
;B[he]
;W[id]
;B[ek]
;W[dl]
;B[dm]
;W[bl]
;B[er]
;W[ki]
;B[mh]
;W[hi]
;B[bs]
;W[ln]
;B[rk]
;W[rl]
;B[lj]
;W[kj]
;B[sf]
;W[rd]
;B[gk]
;W[hk]
;B[gj]
;W[re]
;B[db]
;W[lk]
;B[hj]
;W[ij]
;B[hl]
;W[hm]
;B[ik]
;W[jh]
;B[ii]
;W[jj]
;B[hh]
;W[ei]
;B[gi]
;W[br]
;B[il]
;W[kf]
;B[im]
;W[in]
;B[sl]
;W[sm]
;B[sk]
;W[rn]
;B[gm]
;W[jm]
;B[gn]
;W[cs]
;B[ol]
;W[ld]
;B[mj]
;W[ps]
;B[os]
;W[qs]
;B[op]
;W[pk]
;B[ok]
;W[pj]
;B[pi]
;W[qk]
;B[rj]
;W[po]
;B[le]
;W[ke]
;B[md]
;W[ml]
;B[lc]
;W[lb]
;B[kd]
;W[ie]
;B[hf]
;W[eg]
;B[kc]
;W[kb]
;B[if]
;W[ee]
;B[ef]
;W[de]
;B[dr]
;W[ed]
;B[dc]
;W[fg]
;B[gg]
;W[fe]
;B[gf]
;W[kp]
;B[kq]
;W[jk]
;B[fl]
;W[ip]
;B[iq]
;W[jp]
;B[fh]
;W[eh]
;B[ho]
;W[lo]
;B[mo]
;W[cr]
;B[fi]
;W[ej]
;B[nb]
;W[se]
;B[ib]
;W[ic]
;B[sg]
;W[jd]
;B[ld]
;W[jb]
;B[io]
;W[jn]
;B[ge]
;W[lg]
;B[mg]
;W[li]
;B[ig]
;W[mi]
;B[ni]
;W[ma]
;B[ec]
;W[pm]
;B[lh]
;W[kh]
;B[nm]
;W[mm]
;B[ih]
;W[ds]
;B[es]
;W[mk]
;B[nj]
;W[pe]
;B[oe]
;W[jf]
;B[lf]
;W[kg]
;B[oo]
;W[fj]
;B[mn]
;W[on]
;B[om]
;W[nn]
;B[qe]
;W[pa]
;B[pe]
;W[lq]
;B[mp]
;W[ns]
;B[or]
;W[mr]
;B[nr]
;W[lr]
;B[ms]
;W[pn]
;B[ls]
;W[no]
;B[kr]
;W[na]
;B[oa]
;W[sc]
;B[pl]
;W[cd]
;B[bd]
;W[nd]
;B[oc]
;W[qi]
;B[ri]
;W[ph]
;B[oi]
;W[fk]
;B[ji]
;W[nk]
;B[nl]
;W[nh]
;B[qh]
;W[nf]
;B[ng]
;W[og]
;B[oh]
;W[jg]
)

Game 29.sgf
 --mode gtp --level 12 --boardsize 19 -M 8 --komi 5.5
Size 19
Black = gnugo3321cvs White =
Level black = 12 Level white = 0
Time black = 0 Time white = 1828,0
Score = = W+2.5 (upper bound: 2.5, lower: 2.5)
(
;B[qd]
;W[dd]
;B[cp]
;W[pp]
;B[oc]
;W[fq]
;B[ep]
;W[dr]
;B[fo]
;W[cj]
;B[pn]
;W[nq]
;B[qp]
;W[qq]
;B[qo]
;W[rq]
;B[qj]
;W[jc]
;B[dg]
;W[cm]
;B[gd]
;W[mc]
;B[nn]
;W[ce]
;B[me]
;W[hc]
;B[gf]
;W[hp]
;B[fc]
;W[rd]
;B[re]
;W[qe]
;B[qf]
;W[pe]
;B[rf]
;W[nd]
;B[qc]
;W[of]
;B[br]
;W[ej]
;B[ee]
;W[nh]
;B[je]
;W[ii]
;B[db]
;W[bg]
;B[cc]
;W[gb]
;B[jq]
;W[ir]
;B[rp]
;W[pk]
;B[oj]
;W[mj]
;B[pj]
;W[lm]
;B[ko]
;W[jn]
;B[jr]
;W[gn]
;B[mn]
;W[bo]
;B[lh]
;W[ki]
;B[nl]
;W[iq]
;B[ll]
;W[kl]
;B[jo]
;W[in]
;B[kk]
;W[jk]
;B[jl]
;W[km]
;B[ik]
;W[jj]
;B[il]
;W[ml]
;B[fl]
;W[ph]
;B[fn]
;W[em]
;B[gm]
;W[hm]
;B[fm]
;W[rh]
;B[hl]
;W[hn]
;B[bp]
;W[co]
;B[do]
;W[dn]
;B[gj]
;W[ch]
;B[kd]
;W[kc]
;B[if]
;W[hj]
;B[en]
;W[fp]
;B[ri]
;W[gk]
;B[fk]
;W[gi]
;B[fj]
;W[hk]
;B[fi]
;W[gh]
;B[fh]
;W[gg]
;B[fg]
;W[oo]
;B[ij]
;W[nb]
;B[ob]
;W[nc]
;B[mp]
;W[oa]
;B[pa]
;W[na]
;B[qb]
;W[mm]
;B[nm]
;W[lr]
;B[is]
;W[hs]
;B[js]
;W[hr]
;B[ls]
;W[mr]
;B[mg]
;W[sp]
;B[so]
;W[sq]
;B[ni]
;W[bd]
;B[bc]
;W[ad]
;B[po]
;W[no]
;B[pq]
;W[pr]
;B[op]
;W[np]
;B[mo]
;W[kr]
;B[ks]
;W[ms]
;B[mi]
;W[rn]
;B[qm]
;W[nk]
;B[ro]
;W[oq]
;B[ok]
;W[pl]
;B[pp]
;W[ap]
;B[aq]
;W[ao]
;B[de]
;W[cd]
;B[cg]
;W[bf]
;B[hg]
;W[ld]
;B[le]
;W[li]
;B[md]
;W[ac]
;B[jh]
;W[ji]
;B[cr]
;W[dq]
;B[dp]
;W[kh]
;B[kg]
;W[ds]
;B[ih]
;W[hi]
;B[gl]
;W[bb]
;B[cb]
;W[pd]
;B[ng]
;W[og]
;B[nj]
;W[mk]
;B[lc]
;W[lb]
;B[go]
;W[gp]
;B[mh]
;W[dl]
;B[el]
;W[dm]
;B[sh]
;W[qi]
;B[rg]
;W[dh]
;B[oh]
;W[pg]
;B[pi]
;W[ei]
;B[qh]
;W[id]
;B[od]
;W[ne]
;B[oe]
;W[nf]
;B[pf]
;W[he]
;B[ie]
;W[hd]
;B[fb]
;W[on]
;B[pm]
;W[eq]
;B[ba]
;W[ge]
;B[fe]
;W[hf]
;B[gc]
;W[hb]
;B[ab]
;W[ff]
;B[ef]
;W[eh]
;B[eg]
;W[om]
;B[ol]
;W[mf]
;B[lf]
;W[ed]
;B[gf]
;W[cq]
;B[bq]
;W[cs]
;B[fd]
;W[bs]
;B[hh]
;W[jg]
;B[ig]
;W[ff]
;B[ga]
;W[ha]
;B[fa]
;W[ek]
;B[ln]
;W[ec]
;B[eb]
;W[kf]
;B[lg]
;W[lp]
;B[kq]
;W[lq]
;B[cf]
;W[gf]
;B[ld]
;W[mq]
;B[kp]
;W[qr]
;B[ar]
;W[as]
;B[eo]
;W[kn]
;B[io]
;W[ip]
;B[ho]
;W[jd]
;B[lo]
;W[jp]
;B[qg]
;W[ja]
;B[im]
;B[jm]
;W[lj]
;B[dc]
)