Problem with scipy.optimize.fmin_slsqp when using very large or very small numbers
I got in trouble with this issue too, but I solved it in my project. I'm not sure if this is a general solution.
The reason was that the scipy.optimize.fmin_slsqp calculated the gradient by an approximate approach when the argument jac is set by False or default. The gradient produced from the approximate approach was not normalized (with large scale). When calculating the step length, large value of gradient would influence the performance and precision of line search. This might be the reason why we got Positive directional derivative for linesearch.
You can try to implement the closed form of the Jacobian matrix to the object function and pass it to the jac argument. More importantly, you should rescale the value of Jacobian matrix (like normalization) to avoid affecting line search.
Best.
Has anybody ever encountered problems with fmin_slsqp (or anything else in scipy.optimize) only when using very large or very small numbers?
I am working on some python code to take a grayscale image and a mask, generate a histogram, then fit multiple gaussians to the histogram.  To develop the code I used a small sample image, and after some work the code was working brilliantly.  However, when I normalize the histogram first, generating bin values <<1, or when I histogram huge images, generating bin values in the hundreds of thousands, fmin_slsqp() starts failing sporadically.  It quits after only ~5 iterations, usually just returning a slightly modified version of the initial guess I gave it, and returns exit mode 8, which means "Positive directional derivative for linesearch."  If I check the size of the bin counts at the beginning and scale them into the neighborhood of ~100-1000, fmin_slsqp() works as usual. I just un-scale things before returning the results. I guess I could leave it like that, but it feels like a hack.
I have looked around and found folks talking about the epsilon value, which is basically the dx used for approximating derivatives, but tweaking that has not helped.  Other than that I haven't found anything useful yet.  Any ideas would be greatly appreciated.  Thanks in advance.
james