[Numpy-discussion] skip lines at the end of file with loadtxt

[Stéfan van der Walt]

On Tue, Oct 25, 2011 at 12:28 PM, Massimo Di Stefano
<massimodisasha@gmail.com> wrote:
> urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data',
> 'file.txt') a = np.loadtxt('file.txt', skiprows=1)
>
> but it fails becouse of the txt description at the end of the file,

It's always hard to stop reading before the end of a file, since we
don't know when that's going to happen; I guess it would require a
buffered approach.

Fortunately, there is an easy workaround in your case.  All the text
lines start with "  ", so simply do:

np.loadtxt("amon.us.long.data", comments="  ")

You should see a speedup of about 3 times over genfromtxt (loadtxt
does much less under the hood).

Stéfan