[SciPy-user] storing numpy arrays

Dav Clark dav@alum.mit....
Thu Mar 12 18:14:07 CDT 2009


On Mar 12, 2009, at 3:57 PM, Robert Kern wrote:

> On Thu, Mar 12, 2009 at 17:50, Michael Hearne <mhearne@usgs.gov>  
> wrote:
>> Conor - Thank you for your suggestion.  I am in fact already using
>> pytables in another part of my application.  However, using  
>> PyTables for
>> this portion is not a desirable option.
>>
>> Perhaps I should rephrase the question:
>> What is the the most efficient way to serialize a numpy array into a
>> structure that _could_ be written to a disk, or stuffed into a  
>> database,
>> etc?
>
> Use numpy.lib.format.{read,write}_array() to use the NPY format, which
> should be the most robust and efficient way to do what you want. I
> have used this to store numpy arrays as MySQL BLOBs before, and it
> worked quite well.

It sounds like you are trying to avoid writing a file in order to get  
your blob?  I don't know if you could provide a "file-like" object  
straight to your database for write_array.  You could certainly use  
the dumps function, which returns a pickled array (which is not as  
efficient), or maybe do something clever storing the array.data (a  
python "buffer"), array.dtype.str and array.shape attributes?  That  
could leave you with a result that works across languages.

Is there a more canonical "modern" (a la np.lib.format.write_array)  
which is equivalent to the old dumps method?

Cheers,
Dav


More information about the SciPy-user mailing list