[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-gnubg] Bug in sigmoid?
From: |
Olivier Baur |
Subject: |
Re: [Bug-gnubg] Bug in sigmoid? |
Date: |
Sat, 19 Apr 2003 16:23:25 +0200 |
Le vendredi, 18 avr 2003, à 23:01 Europe/Paris, Joseph Heled a écrit :
I am not sure about that. I have approximated exp(x) with 1+x, which
is mathematically sound and has an o(x^2) error, which is nice in the
[01] range the approximation is used. I think this is not the case for
1/(1+x)
Yes, the approximation used for exp(x) in the current sigmoid function
is correct and quite clever, since we have exp(x) = exp(X + dx) =
exp(X).exp(dx) = exp(X).(1 + dx) + o(dx^2), which requires only one
exp(X) lookup.
Then, I don't understand what you mean about 1/(1+x)...
Second, we need to re-train (we can't keep 2 nets around for practical
reasons).
Can't agree less :-)
And, apart from not being myself able to train a new net :-) , that's
the reason why I used the current sigmoid function approximation to
build the sigmoid2 lookup table, so we get the same results.
Third, I need to see numbers on how faster this is on non scalar,
regular x86 machine.
I don't know if this is a typo, but please note a "scalar" processor is
what you call a "regular" processor; on the other hand, "non-scalar"
and "super-scalar" refer to vector-computing.
Please note the speed increase I have measured in sigmoid2 (+60%) was
for a regular scalar (ie *non* vector) implementation (on a PPC G4
processor); with a vector implementation of sigmoid2 (on the same
processor), I actually got a whopping +250% speed increase...
So let me know what figures you get on a scalar x86 :-)
-- Olivier