# [Numpy-discussion] Half baked C API?

Ralf Juengling juenglin at cs.pdx.edu
Tue Mar 15 09:13:30 CST 2005

>> Did I overlook anything, or do I really have to re-implement
>> things like 'sum', 'argmax', 'convolve', 'cos' in C?
>
>
> Can you think of a real-life situation where you would want to call
> these from C? Usually, C modules using arrays are written to add
> functionality that can not be expressed efficiently in terms of
> existing array operations. If you want to compose things like sum and
> argmax, you can do that in Python.

Yes. Think of dynamic programming algorithms like forward, backward,
and viterbi for Hidden Markov Models. In this case you cannot avoid
a loop over one axis, yet the code in the loop can be expressed in
a few lines by matrix operations like 'dot', 'sum', 'outerproduct',
'argmax', elementwise multiplication, etc. As an example, the forward
algorithm can be written as

alpha[0] = P_YcS[y[0]]*P_S0
gamma[0] = sum(alpha[0])
alpha[0] /= gamma[0]
for t in xrange(1, T):
P_ScY_1_prev = dot(P_ScS, alpha[t-1])
P_SYcY_1_prev = P_YcS[y[t]]*P_ScY_1_prev
gamma[t] = sum(P_SYcY_1_prev)
alpha[t] = P_SYcY_1_prev/gamma[t]

> Note also that if you do need to call these routines from your C code,
> you can always do so via the generic Python API for calling Python
> functions. However, reimplementing them in C may often turn out to be
> simpler - doing sum() in C is really a trivial piece of work.

Sure, many array functions like 'sum' are easy to implement in C, but
I don't want to, if don't have to for performance reasons. In an ideal
world, I'd spend most of my time prototyping the algorithms in Python,
and then, if performance is a problem, translate parts to C (or hand
them to weave.blitz) with only minor changes to the prototype code.
And if that's still not fast enough, then I'd go and rethink the
problem in C.

Referring to the example above, I'd also want that an optimized BLAS
implementation of 'dot' be used if available, and the scipy_core
substitute version if not.

So yeah, I claim that, to make weave a truly useful tool, all array
functions of the Python API should also be available in the C API.
Maybe having a few specialized versions, e.g., for contiguous arrays
of double floats, would be a good idea, too.

Ralf

>
> --
> ------------------------------------------------------------------------
> -------