Re: Covariance Matrix

[John Darrington]

I've pushed an implementation of CORRELATIONS, along with a new
covariance matrix implementation in src/math/covariance.[ch]

It actually calculates more than the covariance matrix: it also 
calculates the 0th 1st and 2nd moments, since these are necessary
steps to obtain the cov.

It's currently implemented as a single pass algorithm, but it will
be straightforward to change that.  There's no categorical variables, 
or interactions, at present.

I'm fairly confident that it's giving the right values and works good
with both missing values and with non-unity caseweights.  To test the 
former, I tried the examples at 
http://www.ats.ucla.edu/stat/Spss/modules/missing.htm
and compared my results with those published. For the latter,  I have
a self-consistency test in tests/command/correlation.sh

I'd be interested in any comments and suggestions on how to proceed
with generalising the implementation to accept categorical variables.
Once that has been done, I think it would be a good idea to re-implement
T-TEST and ONEWAY using the new module - which I hope would vanquish
several thousand lines of rather messy code.

J'


On Mon, Sep 28, 2009 at 08:10:58PM -0400, Jason Stover wrote:
     Given that I like solution b, how should we proceed?  Do you want to
     erase most or all of covariance-matrix.c?  Should I do it, or avoid
     it?
     
     Either way is OK with me. I think one of the first thing to do is
     get rid of any hashing business. No more interactions, too.
     
     I ask because I was the one who wrote all the crap in
     covariance-matrix.c in the first place, so on one hand, I kind of feel
     responsible, but on the other, writing crap doesn't exactly enhance
     one's confidence or motivation.

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://pgp.mit.edu or any PGP keyserver for public key.

signature.asc
Description: Digital signature