[SciPy-User] Sometimes fmin_l_bfgs_b tests NaN parameters and then fails to converge

Yury V. Zaytsev yury@shurup....
Thu Dec 30 11:47:32 CST 2010

Dear Scipy experts,

I am implementing a proof of concept code for a bounded optimization
problem using Scipy optimization module and particularly fmin_l_bfgs_b
bounded optimizer.

The problem is that my code runs fine on one machine, but on another one
at some point the optimizer passes NaN's as parameter values and then
after some time fails to converge normally (it terminates, but with an

I am very confused about that and puzzled to which extent I can trust
the results if they are so much system dependent. I would appreciate if
the developers could tell me whether this problem is reproducible and
what is the reason for its occurrence. If I can provide any additional
information which would be helpful in diagnosing the issue, please bear
with me.

An extended description of my setup and machines and some test code to
reproduce the problem are presented below:


I have two machines:

1) Development machine: Ubuntu Hardy / 32 bit
Linux mypride 2.6.24-28-generic #1 SMP Wed Nov 24 09:30:14 UTC 2010 i686 GNU/Linux

2) Test machine: Ubuntu Lucid / 64 bit
Linux davis 2.6.32-27-generic #49-Ubuntu SMP Thu Dec 2 00:51:09 UTC 2010 x86_64 GNU/Linux

The versions on Python and Numpy/Scipy stack on both machines are

I run ActiveState Python 2.7.1 installed in a virtualenv, where latest
versions of Numpy and Scipy are installed (using pip install numpy / pip
install scipy, scipy needs a patch that I fished out of the svn in order
to compile). The tests pass on both machines with an exception for
nakagami distribution on (1) that I don't care about. The pre-requisites
were automatically installed as follows:

$ sudo apt-get build-dep python-numpy python-scipy

The patch, test logs, test script and optimization logs are attached.

Please note, that on machine (1) it does 5 extra steps in between trying
out NaN parameters for some reason. In this case, the optimization
converges, but it is not always true for my bigger problem. However, you
see already the problem of having extra iterations taking time which can
lead to problems when the number of parameters is 100+. 

I have literally compared the numbers that come out of my bigger
optimization and they are slightly different, although the difference is
like in 10-th significant digit or so, that's why I didn't attribute any
special meaning to it.

The test script actually implements the simplest function I could think
of (it gives inf if x <= 0 and x + 1 if x > 0, so basically the
optimizer has to get as close to zero as it can).

One obvious difference between the machines is the bitness, but also,
the environment on 32-bit machine is older, so this could be a compiler
or library problem. That's why I am seeking for confirmation from other
users of Scipy possibly running on completely different sets of
libraries and compilers.

Sincerely yours,
Yury V. Zaytsev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: scipy-0.8.0-python-2.7.patch
Type: text/x-patch
Size: 1214 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/scipy-user/attachments/20101230/4c46d5c0/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: machine-1-opt.log
Type: text/x-log
Size: 1199 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/scipy-user/attachments/20101230/4c46d5c0/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: machine-2-opt.log
Type: text/x-log
Size: 1034 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/scipy-user/attachments/20101230/4c46d5c0/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: optimizer_test.py
Type: text/x-python
Size: 328 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/scipy-user/attachments/20101230/4c46d5c0/attachment.py 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: machine-1-tests.log
Type: text/x-log
Size: 1744 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/scipy-user/attachments/20101230/4c46d5c0/attachment-0003.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: machine-2-tests.log
Type: text/x-log
Size: 1013 bytes
Desc: not available
Url : http://mail.scipy.org/pipermail/scipy-user/attachments/20101230/4c46d5c0/attachment-0004.bin 

More information about the SciPy-User mailing list