[Scipy-tickets] [SciPy] #794: linalg.eig segmentation fault on unpickled arrays

SciPy scipy-tickets@scipy....
Thu Feb 26 07:17:07 CST 2009


#794: linalg.eig segmentation fault on unpickled arrays
--------------------------+-------------------------------------------------
 Reporter:  tzito         |        Owner:  somebody
     Type:  defect        |       Status:  new     
 Priority:  high          |    Milestone:  0.8.0   
Component:  scipy.linalg  |      Version:          
 Severity:  critical      |   Resolution:          
 Keywords:                |  
--------------------------+-------------------------------------------------
Comment (by pv):

 I think the rationale looks like this:

 Python's unpickle doesn't align the part of its strings corresponding to
 the array data at 8-byte boundaries. Numpy directly uses the memory given
 by pickle, to avoid a copy (unless the pickled array originally had the
 ALIGNED attribute set). There's no way to tell Python's pickle how it
 should allocate its memory.

 So, a copy *must be made* at some point before passing the array on to
 LAPACK, to ensure a proper alignment. Reusing the old memory via
 realloc+memmove seems quite brittle, especially as the memory is owned by
 a Python string.

 Also, it does not feel correct to make the copy immediately on unpickle,
 to work around a problem in a different place in the code. The LAPACK
 wrappers must be forgiving in what they accept, especially as a temporary
 copy for mis-aligned arrays is probably not very difficult to implement.

 So, I think the only correct fix is to treat mis-aligned arrays similarly
 as discontinuous ones, ie., make a temporary copy before passing them on
 to LAPACK. *In addition to this,* we can think about whether avoiding the
 copy in unpickling is a more significant performance gain than ensuring
 that unpickled arrays are always aligned on itemsize boundaries. (It may
 well be that it is more efficient for all computations to ensure that
 freshly unpickled arrays are aligned. But nevertheless, LAPACK wrappers
 should accept mis-aligned arrays, as those could be produced by the user
 in various ways.)

-- 
Ticket URL: <http://scipy.org/scipy/scipy/ticket/794#comment:8>
SciPy <http://www.scipy.org/>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list