[Numpy-discussion] R: fast numpy.fromfile skipping data chunks

Charles R Harris charlesr.harris@gmail....
Wed Mar 13 18:40:00 CDT 2013


On Wed, Mar 13, 2013 at 9:54 AM, Andrea Cimatoribus <
Andrea.Cimatoribus@nioz.nl> wrote:

> Thanks a lot for the feedback, I'll try to modify my function to overcome
> this issue.
> Since I'm in the process of buying new hardware too, a slight OT (but
> definitely related).
> Does an ssd provide substantial improvement in these cases?
>

It should. Seek time on an ssd is quite low, and readout is fast. Skipping
over items will probably not be as fast as a sequential read but I expect
it will be substantially faster than a disk. Nathaniel's loop idea will
probably work faster also. The sequential readout rate of a modern ssd will
be about 500 MB/sec, so you can probably just divide that into your file
size to get an estimate of the time needed.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130313/d0107725/attachment.html 


More information about the NumPy-Discussion mailing list