Re: Covariance Matrix

[Jason Stover]

On Wed, Oct 07, 2009 at 05:01:37PM +0000, John Darrington wrote:
> On Wed, Oct 07, 2009 at 08:54:40AM -0400, Jason Stover wrote:
> 
>      On Tue, Oct 06, 2009 at 07:43:06PM +0000, John Darrington wrote:
>      >     ....  The loop in get_dim should not be necessary,
>      >     since the total number of categories has already been calculated...
>      
>      Where?
> 
> I should have written this more precisely:
> 
> There exists the *opportunity* to pre-calculate the total number of 
> categories (eg.
> in line 141 of category.c) - This would make the loop in get_dim unnecessary 
> and the
> function a whole lot simpler.

Sort-of: Line 141 of category.c updates the total number of
categories for a single variable: cv->n_categories++;

But that isn't the total number of categories that must be considered
for the covariance matrix. For example, if our dictionary had the
variables v1, v2 and v3, and v1 had n1 categories, v2 had n2
categories and v3 had n3 categories, then our covariance matrix would
need to have (n1 - 1)*(n2 - 1)*(n3 - 1) rows. I guess we could compute
this value in category.c, but what if someone only wanted a covariance
matrix for just v1 and v2? Then the the number of rows necessary would be
(n1 - 1) * (n2 - 1). Or maybe they would want v2 and v3, then we would
need (n2 - 1)*(n3 - 1) rows.

There is no way to know in advance how many rows we would need, unless
we just compute the covariance matrix for all the variables. Which could
be OK, I guess. Did you mean to always compute a covariance matrix
for all the variables in the dictionary?

BTW: Given what I just mentioned about the dimension necessary, that
loop would have to be:

  for (i = 0; i < n_vars; i++)
    {
      if (var_is_numeric (vars[i]))
        {
          dim++;
        }
      else
        {
          dim += cat_get_n_categories (vars[i]) - 1;
        }
    }


...instead of just dim += cat_get_n_categories (vars[i]);