[Numpy-discussion] R: fast numpy.fromfile skipping data chunks

Charles R Harris charlesr.harris@gmail....
Wed Mar 13 18:40:00 CDT 2013

On Wed, Mar 13, 2013 at 9:54 AM, Andrea Cimatoribus <
Andrea.Cimatoribus@nioz.nl> wrote:

> Thanks a lot for the feedback, I'll try to modify my function to overcome
> this issue.
> Since I'm in the process of buying new hardware too, a slight OT (but
> definitely related).
> Does an ssd provide substantial improvement in these cases?

It should. Seek time on an ssd is quite low, and readout is fast. Skipping
over items will probably not be as fast as a sequential read but I expect
it will be substantially faster than a disk. Nathaniel's loop idea will
probably work faster also. The sequential readout rate of a modern ssd will
be about 500 MB/sec, so you can probably just divide that into your file
size to get an estimate of the time needed.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130313/d0107725/attachment.html 

More information about the NumPy-Discussion mailing list