[SciPy-dev] Thoughts on weave improvements

Pearu Peterson pearu at cens.ioc.ee
Tue Feb 12 13:58:52 CST 2002

Eric and Pat,

On Tue, 12 Feb 2002, eric wrote:

> Just to make sure we're talking about the same thing, consider that we have
> the function

> the user.  Here is what the C functions will look like  -- the first
> generated by pycod's machinery and the second by the current weave's
> machinery:
> int pure_c_foo(int a, int b)
> {
>     return a+b;
> }


> Generally, f2py calls back into Python into foo.  It could be converted to
> call foo_wrap directly with some savings, but the big win is having Fortran
> functions call pure_c_foo directly so that Python is taken out of the loop
> completely.  f2py will need to ask accelerated objects if the have a pure C
> implementation.  If they do, it'll use this pointer, and everything happens
> as fast as computer-ly possible.  If they do not, they either ask the object
> to compile a new version with the appropriate types, or call the pure python
> version using the current callback scheme.

Ok, very good. So, let me now see how it would work from the f2py point
of view. Here follows four points. The points 2) and 3) are important to
make f2py and weave to work together.

1) Suppose, f2py generates an extension function `gun.run' that wraps the
following Fortran function:

      integer function run(fun)
      external fun
      run = fun(3,4)

In C, this function can be called by name `run_'.
f2py generates the following C/API (pseudo) code:

  PyObject *py_fun;
  int c_fun(int *a,int *b) { 
    /* Note: Fortran expects pointer arguments */
    PyObject *value;
    value = PyObject_CallObject(py_fun,Py_BuildValue("ii",*a,*b));
    return int_from_pyobj(value);

  PyObject f2py_gun_run(PyObject* self,
                        PyObject* args, PyObject* kws
                      ) {
    int ret;
    static char *kwlist[] = {"fun",NULL};


    ret = run_(&c_fun);

    return Py_BuildValue("i",ret);

2) There is one issue. Fortran always expects that arguments are
pointers. Is it possible that weave/pycod could generate the following
  int pure_c_foo(int *a, int *b)
      return *a + *b;
? Otherwise I don't see how I could pass your pure_c_foo to Fortran
functions without additional wrapping. Array arguments must be pointers
anyway, right. Actually, below you see a possible way out of this dilemma
(look at GetAccelerated(py_fun,signature)). Or is it SWIG stuff applicable

3) Assume that issue 2) is solved. Then f2py should generate the following
f2py_gun_run function:

  PyObject f2py_gun_run(PyObject* self,
                        PyObject* args, PyObject* kws
                      ) {
    int ret;
    static char *kwlist[] = {"fun",NULL};

    if (CheckAccelerated(py_fun)) {
       static char *signature = {"int","*int","*int"};
       /* something like
             static int *signature = {INT_ARG,INT_ARR_ARG,INT_ARR_ARG};
          would be more efficent.
       PyObject * accel_fun = GetAccelerated(py_fun,signature);
       ret = run_(&GetAccelerated_Ptr(accel_fun));
    } else if (PyCheck_Function(py_fun)) {
       ret = run_(&c_fun);
    } else {
      // raise TypeError
    return Py_BuildValue("i",ret);

where weave/pycod should provide the following macros
Doesn't matter for f2py how you would call them but it would be
preferable if f2py extension modules would need _not_ to include any
weave/pycod specific header files. For example, the Python equivalent for
CheckAccelerated could be:

  def CheckAccelerated(obj):
      return hasattr(obj,'__weave_accelerated__')

4) To finalize, the Python session would look like the following:

  def foo(a,b):
      return a+b

  import weave
  foo = weave.accelerate(foo)

  import gun
  print gun.fun(foo)   # -> 7

> Not sure yet.  If we find overlap, we should defintely get rid of it
> Let us get a bit farther on merging weave and pycod before we address
> this though.
> I'm still trying to get a handle on how the caching mechanism should work
> for this, and Pat is trying to get Python loop conversion to C working.

Sure. While doing all that it would be great if you could take into
account the issue 2) and provide macros/functions to serve the point in 3).
Then the cooperation between f2py and weave would be almost ideal (the
ideal would be to merge them ;-).


More information about the Scipy-dev mailing list