Thanks! Good to hear an external benchmark. I'll check my implementation again.
Joseph, do you know the stats for gnubg vs pubeval? I'd be most interested in its performance in "expert" mode - ie 0-ply. On Jan 19, 2012, at 5:56 AM, Nikos Papachristou < address@hidden> wrote:
There must be some bug in your implementation. The best neural network that I have trained for Palamedes (including expert features) reaches 0.603 ppg against pubeval. I will be very much surprised if I see anything scoring above 0.75.
I am also curious about the performance of gnubg against pubeval. I would have done it myself if there was an easy way to get gnubg's cubeless evaluations from any backgammon position. Nikos
On Tue, Jan 17, 2012 at 7:28 AM, Mark Higgins <address@hidden> wrote:
How does gnubg perform against the pubeval benchmark in cubeless play?
I ask because I'm playing around with a backgammon network and have got one that wins 83% of games and +0.945ppg against pubeval (10k cubeless games). This is a single 80-hidden-node network with outputs for prob of win, prob of gammon win, and prob of gammon loss; and just the original Tesauro inputs. 0-ply.
But in the TD-Gammon scholarpedia article it says that TD-Gammon 2.1 in 1-ply mode wins only +0.596ppg against pubeval. (I think 1-ply here means the gnubg 0-ply.)
http://www.scholarpedia.org/article/Td-gammon
That seems really low compared to my result, since I'm pretty sure 2.1 had gammon outputs and also extra customized inputs.
So I'm wondering if I'm interpreting this correctly, or if I have an incorrectly-setup version of pubeval, or something like that.
_______________________________________________
Bug-gnubg mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/bug-gnubg
|