[Numpy-discussion] Array data and struct alignment

[Albert Strasheim]

Hello all

I'm busy wrapping a C library with NumPy. Some of the functions operate on a
buffer containing structs that look like this:

struct node {
  int index;
  double value;
};

On the Python side, I do the following to set up my data. examples is a list
containing lists or dicts.

nodes = []
for example in examples:
  if type(example) is dict:
    nodes.append(example.items())
  else:
    nodes.append(zip(range(1, len(example)+1), example))
descr = [('index','intc',1),('value','f8',1)]
self.nodes = map(lambda x: array(x, dtype=descr), nodes)

Assume example = [[1.0, 2.0, 3.0], {4: 4.0}]. The nodes array can now be
accessed in various useful ways:

nodes[0][0] -> (1, 1.0)
nodes[1][0] -> (4, 4.0))
nodes[0]['index'] -> [1,2,3]
nodes[0]['value'] -> [1.0,2.0,3.0])
nodes[1]['index'] -> [4]
nodes[1]['value'] -> [4.0]

On the C side I can now do the following:

PyObject* Svm_GetStructNode(PyObject* obj, PyObject* args) {
   PyObject* op1;
   struct node* node;
   if(!PyArg_ParseTuple(args, "O", &op1)) {
      return NULL;
   }
   node = (struct node*) PyArray_DATA(op1);
   return Py_BuildValue("(id)", node->index, node->value);
}

However, this only works if struct node is tightly packed (#pragma pack(1)
with the Visual C compiler).

I don't know how feasible this is, but it would be useful if NumPy could be
told to pack its data on n-byte boundaries or on "same as the compiler"
boundaries. I realise that there can be problems when mixing code compiled
by more than one compiler, etc., etc., but a simple unit test can check for
this.

Any thoughts?

Regards,

Albert