[Numpy-discussion] ANN: carray 0.3 released

Kevin Jacobs <jacobs@bioinformed.com> bioinformed@gmail....
Thu Dec 23 22:34:07 CST 2010


On Wed, Dec 22, 2010 at 1:58 PM, Francesc Alted <faltet@pytables.org> wrote:

>  >>> %time b = ca.zeros(1e12)
>  CPU times: user 54.76 s, sys: 0.03 s, total: 54.79 s
>  Wall time: 55.23 s
>

I know this is somewhat missing the point of your demonstration, but 55
seconds to create an empty 3 GB data structure to represent a multi-TB dense
array doesn't seem all that fast to me.  Compression can do a lot of things,
but isn't this a case where a true sparse data structure would be the right
tool for the job?  I'm more interested in seeing what a carray can do with
census data, web logs, or somethat vaguely real world where direct binary
representations are used by default and assumed to be reasonable optimal
(i.e., anything sensibly stored in sqlite tables).

-Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20101223/d53aa1cf/attachment-0001.html 


More information about the NumPy-Discussion mailing list