[Numpy-discussion] numpy.loadtext() fails with dtype + usecols

Ryan May rmay31@gmail....
Fri Jul 18 17:16:57 CDT 2008


I was trying to use loadtxt() today to read in some text data, and I had 
a problem when I specified a dtype that only contained as many elements 
as in columns in usecols.  The example below shows the problem:

import numpy as np
import StringIO
data = '''STID RELH TAIR
JOE 70.1 25.3
BOB 60.5 27.9
f = StringIO.StringIO(data)
names = ['stid', 'temp']
dtypes = ['S4', 'f8']
arr = np.loadtxt(f, usecols=(0,2),dtype=zip(names,dtypes), skiprows=1)

With current 1.1 (and SVN head), this yields:

IndexError                                Traceback (most recent call last)

/home/rmay/<ipython console> in <module>()

/usr/lib64/python2.5/site-packages/numpy/lib/io.pyc in loadtxt(fname, 
dtype, comments, delimiter, converters, skiprows, usecols, unpack)
     309                             for j in xrange(len(vals))]
     310         if usecols is not None:
--> 311             row = [converterseq[j](vals[j]) for j in usecols]
     312         else:
     313             row = [converterseq[j](val) for j,val in 

IndexError: list index out of range

I've added a patch that checks for usecols, and if present, correctly 
creates the converters dictionary to map each specified column with 
converter for the corresponding field in the dtype. With the attached 
patch, this works fine:

array([('JOE', 25.300000000000001), ('BOB', 27.899999999999999)],
       dtype=[('stid', '|S4'), ('temp', '<f8')])

Comments?  Can I get this in for 1.1.1?



Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
-------------- next part --------------
A non-text attachment was scrubbed...
Name: loadtxt-dtypes.diff
Type: text/x-patch
Size: 729 bytes
Desc: not available
Url : http://projects.scipy.org/pipermail/numpy-discussion/attachments/20080718/ca8ef292/attachment.bin 

More information about the Numpy-discussion mailing list