[SciPy-User] Catching warning and turning into errors for lmfit / scipy.optimize.leastsq

Matt Newville newville@cars.uchicago....
Fri Apr 5 09:04:10 CDT 2013


Hi Troels,

On Fri, Apr 5, 2013 at 7:18 AM, Troels Emtekær Linnet <tlinnet@gmail.com> wrote:
> Dear Scipy users.
>
> I haven't been dealing with Warning and Errors before, and need a little
> help catching warnings to errors.
>
> I am making some tests on some dataset, which get worse and worse.
>
> I am fitting a decay model, to some intensities.
>
> When the data gets really bad, I start getting some Warnings.
> Getting decay for 8
> Getting decay for 6
> Getting decay for 4
> Getting decay for 2
> TB.py:34: RuntimeWarning: overflow encountered in exp
>   model = amp*exp(-decay*time)
>
> Function is:
> def f_expdecay_lmfit(pars,time,data=None):
>     amp = pars['amp'].value
>     decay = pars['decay'].value
>     model = amp*exp(-decay*time)
>     if data is None:
>         return model
>     return (model-data)
>
> Call is:
> try:
>     lmf = lmfit.minimize(f_expdecay_lmfit, par, args=(datX,
> datY),method='leastsq')
>     fitY = f_expdecay_lmfit(par,datX)
> except (RuntimeError, ValueError, RuntimeWarning, UnboundLocalError) as e:
>     print "Cannot fit expdecay for %s %s. Reason: %s"%(peak, peakname, e)
>     save_the_data_as_null_or_flagged
>
> My problem is, that I don't know how to catch the warnings.
> If I get a warning, I also want to make sure, that this fit is not saved.
>
> I have tried to use the warning package
> import warnings
> warnings.simplefilter('error')
>
> Then I get:
> -------------------
> Traceback (most recent call last):
>   File "lmfit/minimizer.py", line 140, in __residual
>     out = self.userfcn(self.params, *self.userargs, **self.userkws)
>   File "/TB.py", line 33, in f_expdecay_lmfit
>     model = amp*exp(-decay*time)
> RuntimeWarning: overflow encountered in exp
> Traceback (most recent call last):
>   File "TB_fit.py", line 28, in <module>
>     TB.getdecay(XXL,XXL['met'])
>   File "/TB.py", line 268, in getdecay
>     lmf = lmfit.minimize(f_expdecay_lmfit, par, args=(datX,
> datY),method='leastsq')
>   File "lmfit/minimizer.py", line 498, in minimize
>     fitter.leastsq()
>   File "lmfit/minimizer.py", line 369, in leastsq
>     lsout = scipy_leastsq(self.__residual, self.vars, **lskws)
>   File "scipy/optimize/minpack.py", line 283, in leastsq
>     gtol, maxfev, epsfcn, factor, diag)
> minpack.error: Error occurred while calling the Python function named
> __residual
> --------------------------------------------
> If I then rewrite the fitting function also.
>
> def f_expdecay_lmfit(pars,time,data=None):
>     amp = pars['amp'].value
>     decay = pars['decay'].value
>     try:
>         model = amp*exp(-decay*time)
>     except (RuntimeError, ValueError, RuntimeWarning, UnboundLocalError) as
> e:
>         print "Cannot fit expdecay. Reason: %s"%(e)
>     if data is None:
>         return model
>     return (model-data)
>
> I still get:
> ------------------------
> Cannot fit expdecay. Reason: overflow encountered in exp
> Traceback (most recent call last):
>   File "lmfit/minimizer.py", line 140, in __residual
>     out = self.userfcn(self.params, *self.userargs, **self.userkws)
>   File "/TB.py", line 39, in f_expdecay_lmfit
>     return (model-data)
> UnboundLocalError: local variable 'model' referenced before assignment
> Traceback (most recent call last):
>   File "TB_fit.py", line 28, in <module>
>     TB.getdecay(XXL,XXL['met'])
>   File "/TB.py", line 271, in getdecay
>     lmf = lmfit.minimize(f_expdecay_lmfit, par, args=(datX,
> datY),method='leastsq')
>   File "lmfit/minimizer.py", line 498, in minimize
>     fitter.leastsq()
>   File "lmfit/minimizer.py", line 369, in leastsq
>     lsout = scipy_leastsq(self.__residual, self.vars, **lskws)
>   File "scipy/optimize/minpack.py", line 283, in leastsq
>     gtol, maxfev, epsfcn, factor, diag)
> minpack.error: Error occurred while calling the Python function named
> __residual
> -----------------------
>
> I am not able to pass UnboundLocalError.
>
> I want to break out of the fitting rutine for warnings and errors, and save
> that fit as null.
>
> Can someone help me to figure out how to fix this?
>
> Best
> Troels
>
>
>
> Troels Emtekær Linnet
> Ved kløvermarken 9, 1.th
> 2300 København S
> Mobil: +45 60210234
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>


I believe the problem you're having is that the underlying fitting
routine really needs to have a valid array returned to it.   When you
get an RuntimeError exception because the exponentiation gives an
overflow, the variable 'model' is not defined, and your return
statements will cause another exception that the fitting routine can't
handle.  You may find something like this to be helpful:

     try:
         model = amp*exp(-decay*time)
     except (RuntimeError, ValueError, RuntimeWarning, UnboundLocalError) as e:
         print "Cannot fit expdecay. Reason: %s"%(e)
	 return numpy.zeros_like(time)

The other option is to figure why you're getting the overflows and
prevent it.  It looks like you expect "time" and "decay" to be
positive, and so "-decay*time" to be negative.  But numpy.exp()
overflows when it's argument is > 700, which is probably a crazy value
for you.   If "time" is positive-definite, and you expect "decay" to
be positive, you could think about placing a lower bound on "decay" of
"-max(time)/100.", which would allow "decay" to be slightly negative,
but not so much so as to cause the overflow.   That would probably
help avoid the problem in the first place.

Hope that helps,

--Matt


More information about the SciPy-User mailing list