Re: [Bug-gnubg] Luck analysis question

[Joern Thyssen]

On Wed, Jun 11, 2003 at 10:06:14AM -0400, address@hidden wrote
> In a message dated 11/06/2003 15:02:15 GMT Daylight Time, address@hidden 
> writes:
> 
> 
> > yes, but calculating luck adjusted results are much faster than
> > performing many rollouts.
> > 
> > Also, rollouts may still disagree.
> > 
> 
> True but interestingly after rolling out about 30 of the positions with GNU I 
> found no disagreements. There were individual differences in the size of the 
> errors but in total about the same. A small sample only I know but it 
> surprised me none the less.

Yes, but then you still don't know whose the best bot. 

Lets say you found 30 differences. Using rollouts by both snowie and
gnubg we found than gnubg was right in 15 positions and snowie in
the other 15. In order to determine who made the largest errors we need
to know the size of the errors. If the gnubg rollouts says something
different than Snowie's rollouts, then we've made no progress, i.e.,
gnubg says than gnubg lost 0.5 errors and snowie 0.46, but Snowies says
that gnubg lost 0.6 and snowie 0.4. We still do not know who's the best
and by how much.

Jørn