[Help-gsl] Constraints and minimization

[Max Belushkin]

    Hello everyone,

  this post is somewhat related to the thread on multidimensional 
minimization, but not quite, so I decided to start a clean thread on 
this one.

  I am sure the problem of "constraining some values during 
minimization" is well known and often encountered, but the issue is only 
vaguely covered in most articles which employ some methods for this.

  Consider the following problem: the function being minimized, f(x), 
where x is the parameter vector, has several minima. One does not know 
which one is global, but that is a standard problem. However, the 
parameter space must be limited: there exist functions g0(x), ..., 
gN(x), which have upper/lower limits of acceptability. For example, 
g0<=g(x)<=g1 - i.e. a certain value which is a function of x must lie 
within the experimental error.

  One way of constructing a "constraint" (given no analytic solution is 
possible) is to construct a dual chi squared function: C = chi squared + 
constraint. Given chi squared is polynomially bounded, for example, the 
constraint for "g(x) = g0" can look like this: constraint = pow 
(P*[g(x)-g0],2)*exp(pow(P*[g(x)-g0],2).

  Such a constraint is smooth, and the derivative is smooth, thus it 
should not spoil the convergence properties of CG methods. Moreover, it 
contains an adjustable "strength" parameter P, which allows to adjust 
how strong it is relative to the current chi squared, and what dominates 
the "total" chi squared fed to the fit.

  The problem is, it is far from transparent how it influences each 
individual parameter in parameter space, and without any possibility of 
an analytic solution, I do not see how to "map" the constraint over the 
whole parameter space per parameter and determine which areas of that 
space will be suitable to constrain each parameter individually, instead 
of by such a general function which "influences all together, but you 
don't really know how".

  However, this method works for me quite well - convergence without 
constraints or with constraints does not change significantly.

  Has anyone tried other ways of implementing this?

  Thanks!