[NumPy-Tickets] [NumPy] #1665: genfromtxt returns inconsistent output if column converters are used

[NumPy Trac]

#1665: genfromtxt returns inconsistent output if column converters are used
--------------------+-------------------------------------------------------
 Reporter:  dmoore  |       Owner:  somebody      
     Type:  defect  |      Status:  needs_decision
 Priority:  normal  |   Milestone:  2.0.0         
Component:  Other   |     Version:  1.4.1         
 Keywords:          |  
--------------------+-------------------------------------------------------
Changes (by pierregm):

  * status:  new => needs_decision


Comment:

 As you've seen, the output of your converter is not detected as a float,
 but as an object. That's an unfortunate side effect of using a lambda
 function such as yours: what if your input string has only 1 character ?
 You end up taking the float of an empty string, which raises a ValueError.
 In practice, that's exactly what happens below the hood when
 {{{genfromtxt}}} tries to guess the output type of the converter. It tries
 a single value ('1'), fails, and decides that the result must be an
 object... Probably not the best strategy, as it crashes in your case. But
 yours is a buggy case anyway.

 Try that instead of your lambda function
 {{{
 def func(s):
     try:
         r = float(s[1:])
     except ValueError:
         r = 1.
     return r
 }}}

 You could object that as the dtype is defined, it should take precedence
 over the output typeof the converter. Well, I assumed exactly the
 opposite: if the user took the time to define a converter, we should
 respect his/her choice and overwrite the dtype.

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1665#comment:1>
NumPy <http://projects.scipy.org/numpy>
My example project