[SciPy-User] genfromtxt with missing fields
Fri Jun 18 15:49:14 CDT 2010
> Since you can specify which columns to skip, then there is a newline
> character only if it's the last (few) column(s) that's missing
> correct? If not the last column then it's just \t\t? My suggestion
> would be to iterate through the ASCII file, split each line on the
> delimiter, and add tabs for these last missing entries if needed, if I
> understand you correctly.
Yes, I can re-process the data inside my script, but I was hoping there
was some clever numpy (or scipy.io) oriented way to deal with this, and
I should have read the exception better and looked at the full docs for
genfromtxt. The answer is right there: the boolean flag "invalid_raise"
will cause malformed lines to be skipped. The exception even shows
where this will be applied.
numpy/lib/io.py in genfromtxt(fname, dtype, comments, delimiter,
skiprows, skip_header, skip_footer, converters, missing, missing_values,
filling_values, usecols, names, excludelist, deletechars, autostrip,
case_sensitive, defaultfmt, unpack, usemask, loose, invalid_raise)
1319 # Raise an exception ?
1320 if invalid_raise:
-> 1321 raise ValueError(errmsg)
1322 # Issue a warning ?
More information about the SciPy-User