[Octave-bug-tracker] [bug #60348] [octave forge] (statistics) logistic_regression

[Nicholas Jankowski]

Follow-up Comment #6, bug #60348 (project octave):

ok. looking through the algorithm a bit more.  some notes:

1 - right at the start the function performs x = -x.  in the optimization
routine that evaluates gamma, it expands g = [z,x]*[theta;beta], where z is a
unit valued matrix with a column for each discrete response level in Y. z and
z1 basically serve as a logical index arrays for low and high values.  so that
equation is a row-select matrix creating "theta_i + (-x) * beta" for each
level i and value of x. This of course does match the original help text. It
hints as to why the algorithm solves for theta to have an opposite sign of
beta, but removing that sign flip doesn't restore the standard definition, as
x gets used elsewhere in the gradient calcs.  will continue to look and see if
it's easy enough to restore to the standard definition.

2 - returned p is not gamma.  it's a level dependent array of the values used
in the log loss calculation. essentially for a 2-level (0-1) fit it's an array
of:

y = [0 ,   p = [1-px
     1]         px]


or, it's the likelihood of a correct prediction for that input value.  

as you said, the doc states 'P holds estimates for the conditional
distribution of Y given X'. my understanding is that that indicates it should
be just px, or 'gamma' as it's generally given.

e.g., with my trivial comment #5 example, the result is:


>> [theta beta dev d1 d2 p] = logistic_regression (y=[0 0 0 0 1 1 1 1 1
1]',x=[0:9]')
theta = 98.193
beta = 28.047
dev = 3.2503e-06
d1 =

  -4.6993e-08
   9.7705e-07

d2 =

  -1.6251e-06   5.7115e-06
   5.7115e-06  -2.0479e-05

p =

   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000

>> gamma = (1-y).*(1-p) + y.*p
gamma =

        0
        0
        0
   0.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000





    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?60348>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/