[SciPy-User] genfromtxt with missing fields

Skipper Seabold jsseabold@gmail....
Fri Jun 18 15:38:28 CDT 2010

On Fri, Jun 18, 2010 at 4:30 PM, Ian Stokes-Rees
<ijstokes@hkl.hms.harvard.edu> wrote:
> I have an ASCII file with missing fields.  The entries are tab
> delimited, and of mixed type.  The missing field issue is somewhat
> different from what "missing_values" and "filling_values" supports: if a
> field is missing, it marks the end of the entry (i.e. there is a new
> line character).  I cannot work out how to import this.   The command
> (an example below) works fine when I remove the fields with missing
> data.  Any suggestions on how to cope with this would be greatly
> appreciated.
> Ian
> d = genfromtxt('3cdx.snapshot.dat',
> delimiter="\t",
> dtype=[('mtz','S4'),
>    ('scop', 'S6'),
>    ('result', 'S12'),
>    ('start','i4'),
>    ('runtime', 'i2'),
>    ('exitcode','u1'),
>    ('rfz', 'f4'),
>    ('tfz','f4'),
>    ('pak','u1'),
>    ('llginitial', 'i2'),
>    ('llg', 'i2')],
> usecols=(1,3,4,6,7,10))

Since you can specify which columns to skip, then there is a newline
character only if it's the last (few) column(s) that's missing
correct?  If not the last column then it's just \t\t?  My suggestion
would be to iterate through the ASCII file, split each line on the
delimiter, and add tabs for these last missing entries if needed, if I
understand you correctly.


More information about the SciPy-User mailing list