Re: [Help-gsl] Question about Multi-fit linear regression

[Martin Jansche]

Suppose X is your input matrix, then perform SVD to obtain X = U S
V^T.  You can then compute the pseudo-inverse X^+ of X as (V S^{-1})
U^T.  In practice, you probably want to treat the computation of (V
S^{-1}) specially, where you scale the columns of V by the reciprocal
of the appropriate entry of S only if that reciprocal doesn't
overflow; and if you need to regularize, you'd use s/(s^2 + r) to
scale the column where s is the appropriate singular value and r is a
global regularization parameter. Once you have (V S^{-1}), if you
really need the pseudo-inverse X^+, you have to do dgemm(); but since
one usually needs the product X^+ y, it's better to compute (U^T y)
with dgemv() and then do another dgemv() with (V S^{-1}) on the left.
Look at the source of gsl_multifit_linear_svd() for further details.

-- mj

On 9/13/07, Julian Winter <address@hidden> wrote:
> Brian,
>
> Thanks for your response.  Since I'm not a matrix algebra expert, can you
> explain your solution a bit more, perhaps with an example?
>
> Thanks,
> Julian
>
> -----Original Message-----
> From: Brian Gough [mailto:address@hidden
> Sent: Thursday, September 13, 2007 7:06 AM
> To: Julian Winter
> Cc: address@hidden
> Subject: Re: [Help-gsl] Question about Multi-fit linear regression
>
> At Mon, 10 Sep 2007 09:29:47 -0400,
> Julian Winter wrote:
> > I am currently using GSL to perform regression for forecasting,
> > specifically, multi-variate linear regression.  I have tried using the
> > gsl_multifit_linear function, and I am happy with its outputs,
> > however, since it employs singular value decomposition (SVD), it has
> > the limitation that M >= N.  Are there any other functions in GSL that
> > can perform similar regression without limiting the number of columns that
> can be input?
>
> There are no functions for doing underdetermined regression specifically,
> but gsl_linalg_SV_decomp on the transpose of the matrix gives all the
> information needed to compute the pseudo-inverse.
>
> --
> Brian Gough
> (GSL Maintainer)
>
> Network Theory Ltd,
> Commercial support for GSL --- http://www.network-theory.com/gsl/
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.485 / Virus Database: 269.13.15/1003 - Release Date: 9/12/2007
> 10:56 AM
>
>
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.5.485 / Virus Database: 269.13.16/1005 - Release Date: 9/13/2007
> 11:45 AM
>
>
>
>
> _______________________________________________
> Help-gsl mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/help-gsl
>