[Scipy-tickets] [SciPy] #1235: scipy.optimize.leastsq MemoryError

SciPy Trac scipy-tickets@scipy....
Fri Jul 16 09:44:10 CDT 2010


#1235: scipy.optimize.leastsq MemoryError
-----------------------------------------------+----------------------------
 Reporter:  matador                            |       Owner:  somebody
     Type:  defect                             |      Status:  new     
 Priority:  normal                             |   Milestone:  0.9.0   
Component:  scipy.optimize                     |     Version:  0.7.0   
 Keywords:  MemoryError large Jacobian Matrix  |  
-----------------------------------------------+----------------------------
 Hi,

 context:
 I am trying to use the levenberg-marquard algorithm from
 scipy.optimize.leastsq.
 My target function, is very high dimensionnal, and so the Jacobian (which
 I provide on my own).

 Bug/problem statement:
 After 5 iterations I get a MemoryError and indeed, all the memory (16GB)
 and swap(20GB) available got occupied.

 Remarks:
  - Unfortunately I can't contribute my code, because it's a very large
 project (mixing python and c++ via boost::python)
  - I can still give you a snapshot of my script:

 {{{


 def f(x)
    ''' the residual function (target) return a np.array of size 500 000
 floats'''
    ...

 def Dfun(x):
    ''' return a np.array of shape (500000, 50) and dtype np.float32 (to
 save a bit of memory)'''
    ...


 x0=[...]## some initial list of size 50
 # x, success=leastsq(f, x0, maxfev=10, Dfun=Dfun) ## gets a MemoryError

 ## a simple test to check that f and Dfun are not responsible for the
 memory increase (the memory footprint remains constant along the iteration
 (checking that with top command)
 for i in xrange(50):
     print 'iter %i' %(i,)
     res=f(x0)
     jac=Dfun(x0)


 }}}



 I am not sure if this is a bug or a misusage of leastsq from my part. I am
 not sure either if this is the right place to inform of this trouble.
 Any help would be appreciated. I will dig further on my own, by the mean
 time.
 Cheers,
 Loïc

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1235>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list