[Numpy-discussion] Multi thread loading data

Mag Gam magawake@gmail....
Wed Jul 1 06:57:15 CDT 2009


Is it possible to use loadtxt in a mult thread way? Basically, I want
to process a very large CSV file (100+ million records) and instead of
loading thousand elements into a buffer process and then load another
1 thousand elements and process and so on...

I was wondering if there is a technique where I can use multiple
processors to do this faster.

TIA


More information about the Numpy-discussion mailing list