[Scipy-tickets] [SciPy] #1235: scipy.optimize.leastsq MemoryError
SciPy Trac
scipy-tickets@scipy....
Fri Jul 16 09:44:10 CDT 2010
#1235: scipy.optimize.leastsq MemoryError
-----------------------------------------------+----------------------------
Reporter: matador | Owner: somebody
Type: defect | Status: new
Priority: normal | Milestone: 0.9.0
Component: scipy.optimize | Version: 0.7.0
Keywords: MemoryError large Jacobian Matrix |
-----------------------------------------------+----------------------------
Hi,
context:
I am trying to use the levenberg-marquard algorithm from
scipy.optimize.leastsq.
My target function, is very high dimensionnal, and so the Jacobian (which
I provide on my own).
Bug/problem statement:
After 5 iterations I get a MemoryError and indeed, all the memory (16GB)
and swap(20GB) available got occupied.
Remarks:
- Unfortunately I can't contribute my code, because it's a very large
project (mixing python and c++ via boost::python)
- I can still give you a snapshot of my script:
{{{
def f(x)
''' the residual function (target) return a np.array of size 500 000
floats'''
...
def Dfun(x):
''' return a np.array of shape (500000, 50) and dtype np.float32 (to
save a bit of memory)'''
...
x0=[...]## some initial list of size 50
# x, success=leastsq(f, x0, maxfev=10, Dfun=Dfun) ## gets a MemoryError
## a simple test to check that f and Dfun are not responsible for the
memory increase (the memory footprint remains constant along the iteration
(checking that with top command)
for i in xrange(50):
print 'iter %i' %(i,)
res=f(x0)
jac=Dfun(x0)
}}}
I am not sure if this is a bug or a misusage of leastsq from my part. I am
not sure either if this is the right place to inform of this trouble.
Any help would be appreciated. I will dig further on my own, by the mean
time.
Cheers,
Loïc
--
Ticket URL: <http://projects.scipy.org/scipy/ticket/1235>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.
More information about the Scipy-tickets
mailing list