Re: [Bug-gnubg] Gap in guestimated ratings?

[James F. Kibler]

Joern Thyssen wrote:
> 
> On Sat, Jan 25, 2003 at 10:24:34AM -0500, James F. Kibler wrote
> >
> >
> > Joern Thyssen wrote:
> > >
> > > On Fri, Jan 24, 2003 at 01:13:21PM -0500, James F. Kibler wrote
> >
> [snip]
> > I see that the gap is caused by the slight error in the exponential
> > interpolation for error rates above 0.030.  If you remove the factor of
> > 10.0 on the rating, the gap will disappear.
> 
> The formulae should be:
> 
> return 500.0f + 1000.0f * exp ( 0.30f - 10.0 * r );
> 
> The factor of 10 is introduced to decay faster towards 500. With a
> factor of 1 an error rate of -1 (which is almost impossible to obtain)
> will give you an rating of 867. With a factor of 10 you'll get a rating
> of 500.
> 
> However, it's still guestimates :-)

Yes, this correction (changing the 0.03 to 0.30) is much better.  It
removes the gap, yet still pushes bad decisions toward 500. 

> > One might wonder why you resort to exponential interpolation since
> > presumably the error rate per decision cannot exceed 1.
> 
> The error rate per decision can easily exceed 1 (for example, if you
> cube decisions are really lousy).

So, I guess this means that the 'error rate per decision' is really
'error rate per move'?  Do you count an error for each checker movement
and an error in the cube decision in the calculation, yielding a maximum
of 3 errors per move?

> > The lower bound of the linear interpolation could have been set to 1
> > and just  do linear interpolation throughout the range?
> >
> > As an aside, the error rates are reported in the match statistics as a
> > negative number.  The sign must be changed before the rating function is
> > called?
> 
> Yes, although is the other way around: gnubg changes the sign when
> writing the match statistics.
> 
> > What is the second argument (n) to the rating function?
> 
> The match length. The current interpolation scheme does not use it.

If the rating were a function of match length, any idea what the
relationship might be?  Guess I should start tracking match length as
well.  

> > Even with this correction, the ratings seem to be biased high relative
> > to FIBS.  I'll continue to collect some data to improve the guestimate,
> > but it will take me a while to reach a gazillion samples.
> 
> :-)
> 
> Jørn

I currently have about 500 FIBS match logs.  Is there a batch procedure
to allow gnubg to analyze all of them at one time, hopefully producing a
text file as output from each match analysis?

Thanks, Jim