[NumPy-Tickets] [NumPy] #1665: genfromtxt returns inconsistent output if column converters are used

[NumPy Trac]

#1665: genfromtxt returns inconsistent output if column converters are used
--------------------+-------------------------------------------------------
 Reporter:  dmoore  |       Owner:  somebody
     Type:  defect  |      Status:  new     
 Priority:  normal  |   Milestone:  2.0.0   
Component:  Other   |     Version:  1.4.1   
 Keywords:          |  
--------------------+-------------------------------------------------------
 ''Disclaimer: This was tested on 1.4.1 I don't know if it was addressed in
 1.5 (I suspect not based on mailing list correspondence).''

 Simple example of the issue:

 {{{
 >>> import numpy, StringIO
 >>>
 >>> s=StringIO.StringIO('q1,2\nq3,4')
 >>> a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda
 s:float(s[1:])})
 >>> a
 array([(1.0, 2.0), (3.0, 4.0)],
       dtype=[('f0', '|O4'), ('f1', '<f8')])
 }}}

 Notice that this is a 1d array of tuples. Since the data are effectively
 the same type, it would have been preferrable to receive 2d output. Note
 that specifying the dtype does not help:

 {{{
 >>> s=StringIO.StringIO('q1,2\nq3,4')
 >>> a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda
 s:float(s[1:])},dtype=float)
 >>> a
 array([(1.0, 2.0), (3.0, 4.0)],
       dtype=[('f0', '|O4'), ('f1', '<f8')])
 }}}

 dtype appears to be ignored.

 Strangely, however, if the data column could be interpreted without the
 converter, then the output is produced as desired even if the converter is
 used:

 {{{
 >>> s=StringIO.StringIO('1,2\n3,4')
 >>> a=numpy.genfromtxt(s,delimiter=',',converters={0:lambda s:float(s)})
 >>> a
 array([[ 1.,  2.],
        [ 3.,  4.]])
 }}}

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1665>
NumPy <http://projects.scipy.org/numpy>
My example project