[Numpy-discussion] cannot set dtype on record array with 04 datetime records

[John Hunter]

I use record arrays extensively with python datetimes, which works if
you pass in a list of lists of data with the names.  numpy can
accurately infer the dtypes and create a usable record array.  Eg,

    import datetime
    import numpy as np
    rows = [ [datetime.date(2001,1,1), 12, 23.],
             [datetime.date(2002,1,1), 10, 13.],
             [datetime.date(2003,1,1), -2, 1.],
             ]

    r1 = np.rec.fromrecords(rows, names='a,b,c')

    print r1.dtype

prints out: [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]

but if I want to speed things up by providing the dtype, numpy raises
ValueError:

    dtype = [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]

    r2 = np.rec.fromrecords(rows, dtype=dtype)

/home/titan/johnh/test.py
     12 dtype = [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]
     13
---> 14 r2 = np.rec.fromrecords(rows, dtype=dtype)
     15
     16

/home/titan/johnh/dev/lib/python2.4/site-packages/numpy/core/records.pyc
in fromrecords(recList, dtype, shape, formats, names, titles, aligned,
byteorder)
    610
    611     try:
--> 612         retval = sb.array(recList, dtype=descr)
    613     except TypeError:  # list of lists instead of list of tuples
    614         if (shape is None or shape == 0):

ValueError: Setting void-array with object members using buffer.
WARNING: Failure executing file: <test.py>


Running from svn HEAD:

In [2]: numpy.__version__
Out[2]: '2.0.0.dev8480'


Here is a complete script::
import datetime
import numpy as np
rows = [ [datetime.date(2001,1,1), 12, 23.],
         [datetime.date(2002,1,1), 10, 13.],I use record arrays
extensively with python datetimes, which works if you pass in a list
of lists of data with the names.  numpy can accurately infer the
dtypes and create a usable record array.  Eg,

    import datetime
    import numpy as np
    rows = [ [datetime.date(2001,1,1), 12, 23.],
             [datetime.date(2002,1,1), 10, 13.],
             [datetime.date(2003,1,1), -2, 1.],
             ]

    r1 = np.rec.fromrecords(rows, names='a,b,c')

    print r1.dtype

prints out: [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]

but if I want to speed things up by providing the dtype, numpy raises
ValueError:

    dtype = [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]

    r2 = np.rec.fromrecords(rows, dtype=dtype)

/home/titan/johnh/test.py
     12 dtype = [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]
     13
---> 14 r2 = np.rec.fromrecords(rows, dtype=dtype)
     15
     16

/home/titan/johnh/dev/lib/python2.4/site-packages/numpy/core/records.pyc
in fromrecords(recList, dtype, shape, formats, names, titles, aligned,
byteorder)
    610
    611     try:
--> 612         retval = sb.array(recList, dtype=descr)
    613     except TypeError:  # list of lists instead of list of tuples
    614         if (shape is None or shape == 0):

ValueError: Setting void-array with object members using buffer.
WARNING: Failure executing file: <test.py>


Running from svn HEAD:

In [2]: numpy.__version__
Out[2]: '2.0.0.dev8480'


Here is a complete script::
import datetime
import numpy as np
rows = [ [datetime.date(2001,1,1), 12, 23.],
         [datetime.date(2002,1,1), 10, 13.],
         [datetime.date(2003,1,1), -2, 1.],
         ]

r1 = np.rec.fromrecords(rows, names='a,b,c')

print r1.dtype

dtype = [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]

r2 = np.rec.fromrecords(rows, dtype=dtype)


         [datetime.date(2003,1,1), -2, 1.],
         ]

r1 = np.rec.fromrecords(rows, names='a,b,c')

print r1.dtype

dtype = [('a', '|O4'), ('b', '<i4'), ('c', '<f8')]

r2 = np.rec.fromrecords(rows, dtype=dtype)



I filed a ticket on the tracker:

  http://projects.scipy.org/numpy/ticket/1544

Thanks!
JDH