[Numpy-discussion] Coding questions for a type change function

Chris Barker chrishbarker at home.net
Thu Oct 4 16:58:02 CDT 2001


HI all,

I've written a function (in C) that changes the type of an array, in
place, using the same data block. The reason I need this is because I am
reading in a large block of data from a file that is essentially a set
of records, each one a C struct that has a couple of longs and a char in
it. I read it in as a big array, slice iot to separate the fields, and
then need to change the type. I had been using:

A = fromstring(A.tostring(),Int32)

This works fine, but needs a lot of memory, and it kind of slow
(althought enough faster than the file read that it matters little) The
array can be on order of 10MB however, so the memory issue is a problem.
Anyway, I wrote a function that changes the type of data in an array,
without making any copies of the data. It seems to work but I have a few
questions (code to follow):

1) Is there an easier way to do this? I just notices that PyArray_As1D
takes a type as an input... would it do what I want??

2) Is there a way , at the C levbel to reshape an array. PyArray_Reshape
takes a PyObject, which is kind of awkward, I'd love to pass in a few
integers, or maybe a nd, and a pointer to a dimensions array.

3) What I did do is use PyArray_DescrFromType() to create a new
desciption, and then assign the description pointer of the array to the
new one. This seems to work, but I expect that the old description
memory never gets freed. How do I free it (can you tell I'm a C
newbie?). Note that I have called this function is a loop thousands of
times, and not seen memory use go up, but it's a pretty small structure.

4) If I call PyArray_DescrFromType() with an invalid typcode, I get a
crash. How can I check if the typecode is valid??


The code follows, I'd love any feedback anyone can offer.

-Chris


static PyObject * NumericExtras_changetype(PyObject *self, PyObject
*args) {

    PyArrayObject *Array;

    //int *new_strides;
    //int *new_dimensions;

    PyArray_Descr *NewDescription;

    int TotalBytes;
    int NewElementSize;

    char typecode;

	if (!PyArg_ParseTuple(args, "O!c",
                          &PyArray_Type, &Array,
                          &typecode)) {
      return NULL;
    }
    
    if (!PyArray_ISCONTIGUOUS(Array)){
      PyErr_SetString (PyExc_ValueError,
                       "m must be contiguous");
      return NULL;
    }     

    TotalBytes = PyArray_NBYTES(Array);


    // This will crash if an invalid typecode is passed in.
    NewDescription = PyArray_DescrFromType(typecode);

    NewElementSize = NewDescription->elsize;
    
    if (TotalBytes % NewElementSize > 0){
      PyErr_SetString (PyExc_ValueError,
                       "The input array is the wrong size for the
requested type");
      return NULL;
    }    

    Array->descr = NewDescription;
    // does the old descr need to be freed?? how??

    Array->nd = 1;
    Array->strides[0] = NewElementSize;
    Array->dimensions[0] = TotalBytes / NewElementSize;
    
    return Py_BuildValue("");
}









-- 
Christopher Barker,
Ph.D.                                                           
ChrisHBarker at home.net                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------




More information about the Numpy-discussion mailing list