|
From: | Nis Jorgensen |
Subject: | Re: [Bug-gnubg] Confidence intervals from rollouts |
Date: | Thu, 05 Sep 2002 11:00:56 +0200 |
--On 04 Sep 2002 08:22 -0700 David Montgomery <address@hidden> wrote:
Consider a single position, for which we have 3 rollout samples A, B, and C. The idea of rotating the first ply or two is that the variance of the *difference* between two plays should be reduced, since one aspect of the luck has been eliminated.
Permission to disagree, sir. I lack the understanding of exactly what you mean above, but I do understand that I disagree on several levels.
The idea of rotating the first ply or two should be to reduce the difference between the "true" value of a position and the results of rollouts. The variance (or rather standard error) is just a _measure_ of how much we trust the result, and reducing the value is not a goal in itself.
This is very important to stress, especially in cases like this, where we should expect the standard error to go _up_ even though the actual trustworthyness of the rollout should improve
Also, I don't understand why you bring "different plays" into this. This is of course relevant for the "duplicate dice" evaluation, but not for the rotation (for which I would reserve the word "stratification").
So, for example, we would expect that the standard error of abs(A - B) would be less than sqrt(2)*[true standard error of rollout of that size of that position].
I am not sure what you mean by the "standard error of abs(A-B)". I assume you just mean Abs(A-B)?
So I think what I did was to consider that from C I could get an unbiased estimate of this true standard error, and then I compared this (* sqrt(2)) with abs(A-B). I repeated with C-A vs B and C-B vs A, and repeated the whole thing for hundreds or thousands of positions.
Hmmm - I think I start to see what you are doing here. I am not convinced that this is a good way to estimate the value of stratification.
I think I will do some simple coin experiments, and see if this brings me any insights.
-- Nis Jorgensen Greenpeace Amsterdam
[Prev in Thread] | Current Thread | [Next in Thread] |