Re: [Bug-gnubg] Gap in guestimated ratings?

[James F. Kibler]

Joern Thyssen wrote:
> 
> On Fri, Jan 24, 2003 at 01:13:21PM -0500, James F. Kibler wrote

> > Well, I could understand if the gnubg rating was biased relative
> > to the FIBS rating, but how could it have a 200 point gap in the
> > middle?  Gnubg says I'm either pretty good or very bad, not the
> > middle-of-the-road player that I really am!
> >
> > Can you tell me what factors are used in gnubg to estimate the rating?
> > I was thinking to collect some more data and do a least squares fit to
> > the estimator in an attempt to improve the coefficients.
> 
> I call the ratings "guesstimates" because that's what they are!
> 
> The guestimated rating is some simple interpolation formulae based on
> your error rate per decision expressed in normalised equity (see below).
> 
> If you can provide some better interpolation formulae I'll be happy to
> implement them. I'm basically looking for a gazillion matches played on
> FIBS where the error rate per decision is recorded for both players as
> well as their FIBS rating.
> 
> Jørn
> 
> extern float
> absoluteFibsRating ( const float r, const int n ) {
> 
>   static float arRating[] =
>     {  500, 1500,  1600,  1750,  1850,  1950,  2050, 2200 };
>   static float arErrorRate[] =
>     { 1e38, 0.030, 0.025, 0.020, 0.015, 0.010, 0.005, 0.0 };
>   int i;
>   float a;
> 
>   if ( r < 0 )
>     return 2200;
> 
>   /* use linear interpolation between
>      (0.030,1500) and (0.0,2200) */
> 
> for ( i = 6; i >= 1; i-- ) {
> 
>     if ( r < arErrorRate[ i ] ) {
> 
>       a = ( arRating[ i + 1 ] -  arRating[ i ] ) /
>         ( arErrorRate[ i + 1 ] - arErrorRate[ i ] );
> 
>       return arRating[ i ] + a * ( r - arErrorRate[ i ] );
> 
>     }
> 
>   }
> 
>   /* error rate above 0.030 */
> /* use exponential interpolation */
> 
> return 500.0f + 1000.0f * exp ( 0.030f - 10.0 * r );
> 
> }

I see that the gap is caused by the slight error in the exponential
interpolation for error rates above 0.030.  If you remove the factor of
10.0 on the rating, the gap will disappear.

One might wonder why you resort to exponential interpolation since
presumably the error rate per decision cannot exceed 1.  The lower bound
of the linear interpolation could have been set to 1 and just  do linear
interpolation throughout the range?

As an aside, the error rates are reported in the match statistics as a
negative number.  The sign must be changed before the rating function is
called?  What is the second argument (n) to the rating function?

Even with this correction, the ratings seem to be biased high relative
to FIBS.  I'll continue to collect some data to improve the guestimate,
but it will take me a while to reach a gazillion samples.

Thanks, Jim Kibler