[SciPy-dev] Inclusion of cython code in scipy
Wed Apr 23 11:49:22 CDT 2008
On Wed, Apr 23, 2008 at 11:00 AM, Brian Granger <email@example.com> wrote:
> 1) The high-profile scientific computing+python projects that are
> written using it (or pyrex):
> mpi4py (mpi binding in python - currently being rewritten to use cython)
> pytables (hdf5 python bindings)
These are wrappers. Writing wrappers in Cython is a fine idea.
Sage claims over 100K lines of Cython source code, so I agree, they do
have an interest in maintaining Cython. They also have an interest in
getting others to sign on.
Doesn't this wrap the Mersenne Twister PRNG?
> RIght now, the worst case scenario is if cython became unmaintained.
> But, with the large number of people depending on it, I think the
> community would simply pick up the slack - the beauty of open source.
I disagree. It is rare for an open source project to reach a
sustainable level before the original authors depart.
> 2) Cython is unique in what it does.
> All the other solutions for making python code fast (swig, f2py,
> ctypes, boost, etc.) still require spending lots of time writing low
> level C/C++/fortran code. Sure, many of us have the skills to do
> that, but the reality is that few of us have the _time_ to write that
> code. If I had to write the ipython distributed array package in
> C/C++ and then wrap it into python it simply _would_not_happen_
> (because of time constraints).
I don't see how writing Cython is easier than writing C/C++/Fortran.
Clearly if you only know Python, then Cython is an easier language to
learn. However, I doubt the Cython abstraction is so complete that
one can avoid learning C.
You conflate the issues of writing code and wrapping it for Python. I
agree that the latter is a painful excercise, even with tools like
SWIG. I'd happily replace my SWIG wrappers with Cython if the need
arose. However, in the context of larger-scale implementations, I
find it hard to believe that the claimed benefits of Cython outweigh
the concerns I've laid out.
> True, a C/C++ library would live on _if_ these projects vanished. But
> I don't think you have made a strong argument that these projects will
> in fact vanish. Furthermore, I think your actions say the opposite.
> You have invested a huge amount of time working on scipy.sparse (which
> is greatly appreciated). I could be wrong, but I don't think you
> would do that if you really thought scipy was going to die soon :)
Clearly I think SciPy has a long life ahead of it. At the same time,
I spent effort ensuring that sparsetools knows absolutely nothing
about NumPy/SciPy. IMO the utility of a stand-alone library is
greater than a tighter coupling between sparsetools and Python. For
instance, I can now combine the C++ I've written for PyAMG with
sparsetools to make a lightweight algebraic multigrid code in pure
> Another relevant point: recently I have spent quite a lot of time
> wrapping templated C++ code with Cython (a 1 million LOC parallel c++
> electromagnetics code). This code uses the STL extensively and Cython
> has worked really well. If someone really wants to use cython in the
> same way you have used swig, that is completely possible - you still
> end up with the nice C++ library underneath.
As I've said before, using Cython as a wrapper generator is perfectly
reasonable. Using Cython to implement the core functionality of that
1M LOC C++ library would be foolish.
> The big difference though between cython and swig is that the top
> level layer in swig is slow pure python code. In cython, you get fast
> C/python extension types that can be called from other C/C++ extension
> code without going through a slow python layer. That is a huge
Which support the case that Cython is a decent wrapper generator.
Nathan Bell firstname.lastname@example.org
More information about the Scipy-dev