[SciPy-Dev] improvements of optimize package

Christoph Deil deil.christoph@googlemail....
Mon Sep 19 10:41:47 CDT 2011

On Sep 19, 2011, at 4:27 PM, Denis Laxalde wrote:

> On Mon, 19 Sep 2011 08:16:35 -0600,
> Charles R Harris wrote:
>> On Mon, Sep 19, 2011 at 7:17 AM, Denis Laxalde <denis.laxalde@mcgill.ca>wrote:
>>> Charles R Harris wrote:
>>>> On Fri, Sep 16, 2011 at 7:43 PM, Christoph Deil
>>>> <Deil.Christoph@googlemail.com> wrote:
>>>>> - Why not have all optimizers return only two things, x and
>>>>> infodict? The difference to your proposal would be that "ier" and
>>>>> "mesg" as well as solver-dependent extra stuff would all be in
>>>>> infodict instead of having variable-length return tuples, which
>>>>> make it harder to quickly switch optimizers. If it's just a dict I
>>>>> can simply pprint it and as long as I don't look at items that are
>>>>> not available for all solvers, I only have to change one word in
>>>>> the code (the optimizer name) to switch to any other optimizer.
>>>> +1, and put x in the infodict also so you can pickle the whole thing
>>>> in one go or pass it to another function to analyse the results.
>>> Ok. So everything in a dictionary. I guess the 'infodict' name will not
>>> be suitable anymore then. Maybe 'sol'?
>> Just to clarify, I'm suggesting a return like "x, dict", with x in two
>> places.
> Ok. Thanks for clarifying (I misunderstood). I'll keep the 'infodict'
> name then.

I think 'infodict' is fine, although if the dict also contains x, it contains
everything and I would prefer 'result'.

This is what Matt called it in his very nice lmfit package,
although in his case it's actually the Minimizer class itself which stores the fit results:

I have no opinion if (x,result) or only (result) should be returned,
the important point is that all solvers return a same-length tuple and
are thus one can easily learn new ones and try different ones out for a given fit problem.

Looking over your document again, there's only one more name that is not
obvious to me and for which I had to look at the docstring: "ier".
Can we rename it to "status" or if that is too general, maybe "optimizer_status" or something 
similar that is self-descriptive?

It would be amazing if I could do this with any optimizer:

result = <optimizer call>
if result.success:
	print 'Found minimum %s at position %s' % (result.fun, result.x)
	print 'Fit did not converge. The problem was:'
	print result.status, result.message

Specifically I think it would be nice if "success" and "fun" (the function value on optimizer exit)
were added to the results dict for all optimizers,
since these are the most basic fit results besides "x" every user wants.

The current problem with "fun" is that only some optimizers return it,
so for those I have to add a line or two of code to do it myself after the fit.
Since every optimizer has a "fun" value on exit I guess it doesn't hurt
to put it in the results dict for all of them without any performance hit?

The current problem with ier = status is that each optimizer has different numbers for success,
e.g. for leastsq 1,2,3 or 4 is success, for anneal 0 and 1 is success, and looking at the fmin docu I can't
figure out how to tell if the fit succeeded at all.
I guess the numbers returned are the ones from the underlying libraries and thus should not be changed,
so maybe adding a bool success flag to every optimizer for new users would be nice. 

If others like the dot notation lookup too (result.status possible in addition to result["status"]),
it can be added to the results dict using the five lines of code described here:

Denis, thanks a lot for your effort!

More information about the SciPy-Dev mailing list