[Numpy-discussion] reading *big* inhomogenous text matrices *fast*?

[Sebastien Binet]

Hi,

> > Raymond Hettinger had a good talk at PyCon this year about the details
> > of the Python containers. Here are the slides from the EuroPython
> > version (I assume).
> >
> >   http://www.pycon.it/static/pycon2/slides/containers.ppt
>
> Thanks!  Looks like the only caveat is that the whole thing may slow down
> if the reallocation operation itself is very inefficient.  Which probably
> isn't the case with a modern Linux distro and recent libc.  I'm thinking
> whatever went wrong had to be my fault :-)

just thinking out loud...
could the slow down be induced by the memory allocation, triggering many GC 
collections ?
I recall a post on c.l.p where disabling the gc during pickling/unpickling a 
complicated data structure tremendously improved performances...

cheers,
sebastien.
-- 
###################################
# Dr. Sebastien Binet
# Lawrence Berkeley National Lab.
# 1 Cyclotron Road
# Berkeley, CA 94720
###################################