[SciPy-user] What is fastest load/save matrix methods?

Arnd Baecker arnd.baecker at web.de
Mon Dec 19 04:01:37 CST 2005


On Mon, 19 Dec 2005, Hugo Gamboa wrote:

> Hi there,
>
> I need to work with large matrixes that come in ascii format.
>
> I have the functions to build the matrix from the ascii file but it
> takes too long.
> I can spend some time convert ingthis ascii files to some other format
> so that the re-readeing be fastened.
>
> What is the fastest (binary) format readly available in scipy to
> load/save matrixes?

I am not sure if it is *the* fastest way, but have you had a look
at scipy.io?
In particular scipy.io.fwrite and scipy.io.fread:

In [1]:import scipy
In [2]:scipy.io.fwrite?
Type:           builtin_function_or_method
Base Class:     <type 'builtin_function_or_method'>
String Form:    <built-in function fwrite>
Namespace:      Interactive
Docstring:
    numpyio.fwrite( fid, Num, myarray { write_type, byteswap} )

         fid =       open file stream
         Num =       number of elements to write
         myarray =   NumPy array holding the data to write (will be
                     written as if ravel(myarray) was passed)
    OPTIONAL
         write_type = character ('cb1silfdFD') describing how to write the
                      data (what datatype to use)  Default = type of
                      myarray.
         byteswap =   0 or 1 to determine if byteswapping occurs on write.
                      Default = 0.

In [3]:scipy.io.fread?
Type:           builtin_function_or_method
Base Class:     <type 'builtin_function_or_method'>
String Form:    <built-in function fread>
Namespace:      Interactive
Docstring:
    g = numpyio.fread( fid, Num, read_type { mem_type, byteswap})

         fid =       open file pointer object (i.e. from fid =
open('filename') )
         Num =       number of elements to read of type read_type
         read_type = a character in 'cb1silfdFD' (PyArray types)
                     describing how to interpret bytes on disk.
    OPTIONAL
         mem_type =  a character (PyArray type) describing what kind of
                     PyArray to return in g.   Default = read_type
         byteswap =  0 for no byteswapping or a 1 to byteswap (to handle
                     different endianness).    Default = 0.

HTH, Arnd



More information about the SciPy-user mailing list