[NumPy-Tickets] [NumPy] #1708: pickle.loads() of pickled numpy.array can corrupt memory

[NumPy Trac]

#1708: pickle.loads() of pickled numpy.array can corrupt memory
--------------------+-------------------------------------------------------
 Reporter:  quarl   |       Owner:  somebody
     Type:  defect  |      Status:  new     
 Priority:  high    |   Milestone:  2.0.0   
Component:  Other   |     Version:  1.4.1   
 Keywords:          |  
--------------------+-------------------------------------------------------
 We discovered that pickle.loads() on a pickled numpy.ndarray can lead to
 memory corruption.  I can reproduce the problem with arrays of size 1
 (length=1, dtype=bool or byte) pickled using protocols 1 or 2:
 {{{
 pickle.loads(pickle.dumps(numpy.array([True]), protocol=-1))
 }}}

 In my tests, this affects Solaris and Linux, Sparc, x86, and x86_64,
 Python versions up to 2.6.6, numpy version 1.4.1.

 After unpickling the size-1 array into 'data', I replace data[0] with an
 arbitrary byte (0xbb in the example below) and it overwrites some
 unrelated data (regexp buffers in the example below).  If the original
 data contains array([2]) instead of array([1]), then it affects "\x02" in
 the regexp buffer instead of "\x01".

 The second use of pickle.loads() then gets confused.

 {{{
 #!/usr/bin/env python

 import numpy
 import pickle
 import re

 # Unpickle a bool or byte array of length 1:
 data = numpy.array([1], dtype='b') # similar issue with
 numpy.array([True])
 blob = pickle.dumps(data, protocol=1) # protocol 1, -1 exhibit bug, but 0
 is 'safe'
 data = pickle.loads(blob)

 print repr(re.sub("a(.)", "\x01\\1", "a_"))
 # '\x01_' [correct]

 data[0] = 0xbb

 print repr(re.sub("a(.)", "\x01\\1", "a_"))
 # '\xbb_' [?!]

 lose = pickle.loads(blob) # crashes
 }}}

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1708>
NumPy <http://projects.scipy.org/numpy>
My example project