[Help-gsl] Matrix multiplication precision problem ???

[Marco Dinarelli]

Hello everybody,

I'm getting a problem that so far I could not manage to solve,
I'm guessing it's a precision problem based on a past thread I found in the
mailing-list archive, but I'm not completely sure…

Basically I'm writing a neural network language model using gsl.
In the main training function several matrices and vectors are involved.

In order to keep track and check everything I'm doing is correct, I wrote a
first prototype in octave and then I rewrote the same things in C with gsl.

When I run the two programs, at first results are the same, but after some
iterations to train the network, results start to diverge, with the C
implementation having worst results and cross-entropy fluctuating values.

After checking several things, and not understanding the source of the
problem, I wrote the matrices involved in the computations to a file, for
both programs, octave and C versions, thus I can compare intermediate
computation for both programs.
Then I compare the corresponding matrices to check if they are equal (with
gsl_matrix_equal) or not.
Only 2 out of 26 matrix pairs are different, and by very small values,
10e-15 on average over matrix elements, or less.

Please note that the two matrix pairs that are different are computed from
other matrices that are equal in the two program versions, thus that's even
more strange to me.

My question is: Is this a precision problem or there should be a mistake
somewhere ?

Please note also that, although difference is very small, since in a neural
network matrix are, let's say, interdependent, small differences in 2
matrices are propagated, after some iterations of training, to all the
other matrices.
Thus at the end the model trained by the C version is different from the
one trained by the octave version, in particular the former is worst.

So, in case this is a precision problem, does anybody know how to overcome
it ?

Any help would be more than appreciated.
Thanks in advance

Best regards

Marco