[Scipy-tickets] [SciPy] #1649: ndimage convolution with large mask uses too much memory

SciPy Trac scipy-tickets@scipy....
Mon May 7 05:22:27 CDT 2012


#1649: ndimage convolution with large mask uses too much memory
---------------------------+------------------------------------------------
 Reporter:  stefan         |       Owner:  somebody   
     Type:  defect         |      Status:  new        
 Priority:  normal         |   Milestone:  Unscheduled
Component:  scipy.ndimage  |     Version:  0.10.0     
 Keywords:                 |  
---------------------------+------------------------------------------------

Comment(by thouis):

 Based on tracing on my OSX machine using "Instruments",  the problem is in
 this allocation:

 [https://github.com/scipy/scipy/blob/master/scipy/ndimage/src/ni_support.c
 scipy/ndimage/src/ni_support.c]:
 {{{
  NI_InitFilterOffsets(...)
  [...]
   *offsets = (npy_intp*)malloc(offsets_size * footprint_size *
                                                        sizeof(npy_intp));
    if (!*offsets) {
        PyErr_NoMemory();
        goto exit;
    }
    if (coordinate_offsets) {
        *coordinate_offsets = (npy_intp*)malloc(offsets_size * rank *
                                        footprint_size * sizeof(npy_intp));
        if (!*coordinate_offsets) {
  [...]
 }}}

 offset_size is basically footprint_size, so for the first allocation, on
 my 64-bit machine, I'd expect:
 {{{
 >>> 125*125*125*125*8
 1953125000
 }}}

 I see a 1.82 GB object allocated during the example run:
 {{{
 >>> 1.82*(2**30)
 1954210119.68
 }}}

 If the test case somehow triggered the coordinate_offsets allocation as
 well, it would be even worse (total would be 3x as large).

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1649#comment:1>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list