[SciPy-user] empty()

Dinesh B Vadhia dineshbvadhia@hotmail....
Fri Mar 7 16:39:18 CST 2008


Thanks!  What I need to do is 1) create a very large sparse matrix A (with integer 1's and A is very sparse), 2) pickle the matrix and 3) unpickle it when needed in various modules.

Performance of unpickling (ie. step 3) is important as the matrix is very large.  Hence, in step 1, I didn't really want to initialize the matrix A with zeroes() because it is adding unnecessary 'weight' to the pickling/unpickling and hence performance.

Appreciate your thoughts.  Cheers!

Dinesh

...............................................................
Message: 4
Date: Thu, 6 Mar 2008 20:05:50 +0100
From: "Anne Archibald" <peridot.faceted@gmail.com>
Subject: Re: [SciPy-user] empty()
To: "SciPy Users List" <scipy-user@scipy.org>
Message-ID:
<ce557a360803061105o2cd1cdb7s82e7f777c732bafd@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

On 06/03/2008, Dinesh B Vadhia <dineshbvadhia@hotmail.com> wrote:

> I have to ask but in
>  > A= scipy.asmatrix(scipy.empty((I, J), int))
>
> does the empty() function really 'empty' out the array or does it just
> allocate space for the array leaving spurious content in the array?
>
> I want to make sure before starting to use empty() with wild abandon!

empty() creates a fresh new array but does not touch the contents.
Thus they can be anything; for freshly-allocated very large arrays
they may be all zeros on some systems, but otherwise they usually
contain malloc bookkeeping information or old data. Thus the array is
packed with spurious content.

It is probably a good idea to either use
A = numpy.zeros(<whatever>)
or, if you want to be sure to notice if you fail to set some element
of the array,
A = numpy.zeros(<whatever>)/0.0
(this latter ensures that the array is filled with "Not a Number", a
special floating-point value that indicates invalid data). Only if you
find your program has no bugs and is too slow is empty() a good idea.
It's usually also worth trying to construct the array directly with
the correct values - for example using arange() or eye() - rather than
filling in values after the fact.

I should also point out that both empy() and asmatrix() are *numpy*
functions, not scipy. That they are also available in scipy is a
historical quirk; scipy simply reexports some of the numpy functions
in its own namespace (complicating the process of finding the real
scipy functions). It's a better idea to access these functions through
the numpy namespace.

Anne

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/scipy-user/attachments/20080307/9f22930a/attachment-0001.html 


More information about the SciPy-user mailing list